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In This Issue 


This issue of Survey Methodology continues the celebration of 25 successful years that was 
marked by the publication of the December 1999 issue. The first seven papers of this issue, by 
prominent statisticians working in survey methods, were invited to help mark this occasion but were 
not included in the December issue due to space limitations. I would like to extend a special word 
of thanks to all of the authors who helped to make these two celebration issues so special and 
memorable. 

To start off this special issue Kalton reviews developments in survey research over the past 25 
years since Survey Methodology first started publishing. He first describes developments in survey 
taking as a profession, specifically the rise of specialist journals and professional associations for 
survey methodologists, as well as the international and multidisciplinary aspects of the profession. 
He then reviews developments in survey methods including questionnaire design, data collection, 
non-sampling errors, sampling methods and estimation. Finally he discusses the rise in importance 
of panel surveys and international surveys, administrative data sources and analysis of survey data. 

Bellhouse traces the parallel developments in survey taking and computing over the twentieth 
century. He first describes the interaction between census taking and the early development of 
computing machines and digital computers. Later, developments in scientific computation lead to 
the use of more sophisticated statistical methods and models. He concludes his story with 
discussions of the development of statistical software for surveys and of model-related methods. 

Bailar discusses the role of statistics in census taking, with particular emphasis on errors in census 
counts due to census errors of various sorts and adjustment of census counts using sample based 
estimates of net undercount. The various sources of errors in censuses are described. Use of 
statistical methods for census evaluation, quality control in census processing, and imputation is also 
discussed. Using a model for census bias and variance, the potential efficacy of census adjustment 
procedures is illustrated. 

Isaki, Tsay and Fuller consider estimation of census adjustment factors using data from the 1990 
post enumeration survey. Their estimators are based on a components of variance model with a 
fixed linear predictor and a random effect describing the unknown true adjustment factor for each 
of 336 post-strata. They consider alternatives based on using an estimate of the full variance- 
covariance matrix of the direct survey errors of the post-stratum adjustment factors versus using only 
the diagonal elements. Use of the diagonal elements only can reduce the effects of instability in the 
estimate of the full variance-covariance matrix. In an empirical comparison they find that a 
compromise between these two extremes works best. They also restrict the model based adjustment 
factors so that the estimate of total population matches that obtained from the direct survey estimates 
of these adjustment factors. 

Lachapelle and Kerr present an innovative use of a coverage study to examine the demographic 
estimates of population. Their approach decomposes the results from Statistics Canada’s Reverse 
Record Check (RRC) to provide an additional source of data that can be compared to the more 
traditional administrative record based estimates of the components of growth. The objective of this 
comparison is to identify major sources of error in either the administrative record based or the RRC 
estimates. They also show how the error of closure can be decomposed into two parts: differences 
between the RRC and the Census estimates of enumerated population and differences between the 
RRC and administrative record based estimates of growth. 

In their paper Feder, Nathan and Pfeffermann consider repeated sampling from a hierarchical 
population. At each fixed time point the population can be described by a two level model; first and 
second level random effects are then allowed to evolve stochastically over time. In particular, the 
case where second level units remain in the sample for only a few occasions, as for example in many 
labour force surveys, is considered. A two step estimation procedure is proposed. In the first step 
the two-level model is fit to each time point independently to obtain estimates of the fixed effects. 
Time series parameters are estimated in the second step. Sampling weights can be incorporated into 
both steps to account for possibly informative sampling. 

Rivest and Belmonte propose measurement of the mean square error of small area estimators 
conditionally on the realized smoothing model. They propose a natural estimator for this MSE; 
however, the estimator can be quite unstable when there is a lot of smoothing. They also propose 
a correction for bias in the case that the distributions of the direct estimators are skewed. Finally 
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they investigate the properties of their estimator in an Empirical Bayesian context and illustrate their 
method using undercoverage data from the 1991 Canadian Census. 

Shao addresses an important topic - the evaluation of cold deck imputation methods. Since 
computer technology continues to make it easier to store and access data from previous and related 
surveys, imputation methods that make use of this auxiliary data will become increasingly important. 
As a result, Shao takes the first steps in evaluating how various cold deck imputation methods will 
perform relative to other imputation methods. 

Thompson and Frank discuss model based estimation for link-tracing designs. In link-tracing 
designs, links are followed from one respondent to another. Network sampling and snowball 
sampling are just two examples. After a general introduction to the area, they present several link- 
tracing designs. They then present a graphical model for the linked population. Finally they develop 
likelihood based inference procedures for such populations using data from link-tracing designs. 

Théberge attempts to solve the problem of extreme weights due to the calibration estimator by 
relaxing somewhat the calibration equation requirements. In fact, the problem is one of 
minimization similar to that encountered in ridge regression. He also reviews other means of 
restricting weights. He discusses the asymptotic properties of calibrated weights, and provides 
necessary and sufficient conditions for the existence of restricted weights satisfying the calibration 
equation. He also outlines a way of formulating the estimation problem by controlling the 
significance given to the calibration equation, and describes various means of restricting weights that 
do not rely on the use of a specific distance. Finally, he suggests an estimator having restricted 
weights that is useful for small domains, and deals with outliers by developing a method similar to 
that used to handle extreme weights. 

Two short notes conclude this issue. Losinger, Garber, Wagner and Hill present a case study in 
the care that must be taken when adjusting for non response in different waves of a survey. Finally, 
Shaffer looks at the estimation of regression coefficients using survey data when the assumption of 
fixed auxiliary variables is relaxed. 

You may recall that the December issue of Survey Methodology was made available, on an 
experimental basis, in an electronic format on the Statistics Canada web site. There was also a web 
based survey to guage your reactions and preferences with respect to an electronic version of the 
journal. Although there was quite a bit of interest in an electronic version, it seems that the time is 
not yet ripe for publishing electronically on a regular basis. We will certainly be reconsidering this 
option in the near future, and your responses to the survey will help to improve any future electronic 
version. In the meantime, we will continue to publish a print version of the journal for the 
foreseeable future. 


M.P. Singh 
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Developments in Survey Research in the Past 25 Years 


GRAHAM KALTON'’ 


ABSTRACT 


In recognition of Survey Methodology’s silver anniversary, this paper reviews the major advances in survey research that 
have taken place in the past 25 years. It provides a general overview of developments in: the survey research profession; 
survey methodology — questionnaire design, data collection methods, handling missing data, survey sampling, and total 
survey error; and survey applications — panel surveys, international surveys, and secondary analysis. It also attempts to 


forecast some future developments in these areas. 


KEY WORDS: Survey research profession; Survey methodology; Survey applications; Questionnaire design; International 


surveys. 


1. INTRODUCTION 


Survey Methodology is celebrating its silver anniversary 
this year. In recognition of this milestone, this paper aims 
to review the major developments in survey research over 
the past 25 years. I should note, however, that for several 
reasons I shall be somewhat lax in my dating of events. 
First, there was, of course, no watershed in survey research 
in 1975. Rather, many of the major developments over the 
past quarter century built on the foundations laid by earlier 
work. Second, it takes time for many advances in method- 
ology to be fully accepted and adopted. Third, I am using 
as my benchmark a text on survey methodology that Sir 
Claus Moser and I published in the United Kingdom in 
1971 (the second edition of Survey Methods in Social 
Investigation, hereafter referred to as Survey Methods), so 
that my time frame actually extends over 30 years or so. 

The paper reviews the developments in survey methodol- 
ogy, including questionnaire design, survey sampling, data 
collection methods, data processing, and survey analysis. 
Computers will feature prominently in the discussion since 
they have had a major impact on many, but not all, method- 
ological developments. The paper also reviews the effects 
of these methodological developments on the practice of 
survey research, including the growth in panel surveys, 
international surveys, and secondary analysis. The main 
emphasis is on population surveys, but some references are 
also made to establishment surveys. Also, in reflecting my 
experience, the paper will no doubt have a slant toward 
work done in the United States. Before turning to develop- 
ments in survey methods and practice, I will first describe 
the great expansion that has taken place in the number of 
surveys being conducted and the emergence of a clearly 
identified survey research profession. 


2. THE SURVEY RESEARCH PROFESSION 


Most of the history of survey research is contained in the 
twentieth century. The field began to take off in the 1930’s, 


grew considerably during the Second World War, and has 
grown at a substantial rate ever since. By 1975, surveys of 
both households and establishments were well established as 
the means to meet the needs for statistical data of policy- 
makers and researchers on a wide range of subjects, such as 
manufacturing and trade, agriculture, employment and un- 
employment, family expenditure, nutrition, health, educa- 
tion, travel, aging, and crime. In addition, surveys conducted 
by academic and other researchers in sociology, economics, 
political science, psychology, education, social work and 
public health, public opinion and election polls, and market 
research have flourished. The field has continued to expand 
at a rapid rate in the past 25 years, particularly as more 
policymakers have learned to appreciate the value of survey 
data and as advances in survey methods have enhanced the 
ability of survey researchers to respond to the demands for 
statistical data. The continuing demand of policymakers for 
more and more sophisticated data has prompted advances in 
survey methodology and has also led to the solidification of 
a broadly based survey research profession. 

The rapid growth in survey research has come about in 
part because of an expansion in the range of topics that are 
considered suitable for study using survey methods. Adven- 
turesome researchers have constantly and successfully 
challenged the conventional wisdom of their times about the 
subject matters that surveys could cover. These challenges 
have continued during the past 25 years so that there are now 
very few subjects that are ruled out for study in surveys 
based on valid probability samples. Some of the new 
subjects of study are sensitive ones, such as sexual behavior 
and illicit drug use, for which the application of survey 
methods has required the development of special data 
collection techniques. Other new subjects have required the 
incorporation of additional data collection methods, such as 
medical examinations for sampled individuals, videotaping 
of teacher-student interactions in classrooms, and placing 
environmental monitoring equipment in sampled house- 
holds. Tackling more difficult subject matters has been a 
constant stimulus to methodological research. 
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4 Kalton: Developments in Survey Research in the Past 25 Years 


Prior to 1975 there were no widely distributed specialist 
journals in survey methodology. Refereed papers on survey 
methodology were published in a variety of journals. 
Statistical journals published, and continue to publish, 
papers mainly on the more statistical aspects of survey 
research, particularly survey sampling. Journals like Public 
Opinion Quarterly published, and continue to publish, 
papers on survey methodology. Market research journals 
publish papers on survey methodology relevant to market 
research. Journals in various subject-matter disciplines in 
the social sciences, public health, efc., sometimes publish 
papers on survey methods relevant to their disciplines. This 
situation was not ideal since there was no natural outlet for 
some good papers on survey research methods and because 
the literature was widely scattered. The introduction of 
Survey Methodology in 1975 and the Journal of Official 
Statistics in 1985, both now well-established journals, has 
remedied this situation. 


Another notable development has been the establishment 
of professional associations for survey methodologists. For 
example, the International Association of Survey Statis- 
ticians (IASS) was founded in 1975 as a section of the 
International Statistical Institute; the Section on Survey 
Research Methods of the American Statistical Association 
was established in 1978, after being a subsection of the 
Social Statistics Section from 1974 to 1977; and the Social 
Statistics Section of the Royal Statistical Society was formed 
in 1976, initially as the Social Statistics and Survey 
Methodology Study Group. 


In recent years, several of these associations, sometimes 
together with other associations (particularly the American 
Association for Public Opinion Research), have collabo- 
rated to run international conferences on specific topics in 
survey methodology. A special feature of these conferences 
is that many of them have been structured to cover their 
chosen topics in a comprehensive manner so that they could 
generate well-rounded texts. This feature was introduced to 
address the shortage of literature on survey methodology 
that resulted from the fact that survey methodologists are 
practitioners with little time to publish. The result has been 
the production of edited volumes on such topics as panel 
surveys, telephone surveys, business surveys, measurement 
errors in surveys, survey quality, and computer-assisted 
survey information collection. 


Many other conferences on survey methodology have 
also been held in recent years. Some have been organized 
by government agencies, such as Statistics Canada, the U.S. 
Census Bureau and the U.S. Federal Committee on 
Statistical Methodology (also founded in 1975). Others 
have been organized by professional associations, such as 
the ASS and the Association for Survey Computing. The 
proceedings from these conferences, and those of the 
Section on Survey Research Methods of the American 
Statistical Association, contribute greatly to the growth in 
the literature on survey methodology. 


Two other aspects of the development of the survey pro- 
fession deserve comment. One is its internationalism. The 
international conferences described above have led to publi- 
cations with authors from many different countries. 
Although there are cultural differences between countries 
that need to be taken into account in data collection, 
research on survey methodology shares a good deal in com- 
mon across countries. In addition, international surveys are 
becoming more prevalent, with the need to standardize 
procedures across countries (see the discussion below). In 
general, international cooperation in survey research is pro- 
gressing well, but there is one area where much more could 
be done. Like the developed countries, the developing and 
transition countries need statistical data from surveys. 
However, they often lack the necessary expertise. The 
IASS, international agencies like the U.N. Statistical Office, 
a number of government statistical agencies, and a number 
of other bodies make valuable contributions to training 
survey researchers from developing and transition coun- 
tries, but the level of support currently available for this 
training falls far short of what is needed. 

Another noteworthy aspect of the development of the 
survey research profession is its multidisciplinary nature. 
As survey research has become established as a profession, 
it has developed a number of subdisciplines. Thirty or so 
years ago, a survey methodologist might expect to cover all 
aspects of the subject, but that is no longer possible at the 
highest technical level. The statistical level of the tech- 
niques of survey sampling and survey analysis used by 
survey statisticians has advanced greatly, survey methodol- 
ogists are increasingly using theories and techniques from 
sociology, psychology, and anthropology, and computer 
specialists now need to use much more sophisticated 
methods for data capture and processing than in the past. 
This inevitable segmentation of survey methodology as the 
field progresses puts at risk a unified professional identifi- 
cation, particularly since the subdisciplines are each also 
associated with their own different fields. Given the 
importance of interdisciplinary collaboration in survey 
research, mechanisms to foster that collaboration may be 
needed in the future (see also section 5). 

As with the developing and transition countries, the 
developed countries face a shortage of well-trained survey 
statisticians and methodologists. There is the need both to 
attract more people into the profession and to provide more 
training opportunities for them. There are a few graduate 
programs at universities and some faculty who specialize in 
the field, but the numbers are inadequate given the needs. 
The multidisciplinary collaboration involved in constructing 
and conducting a survey implies that the training should 
have a multidisciplinary component, so that the various 
specialists can communicate effectively with one another. 
Moreover, the instructors should include persons with 
practical survey experience. These specifications make it 
even more difficult for a graduate program in survey 
methodology to be mounted in most universities. An 
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alternative approach is that adopted by the Joint Program in 
Survey Methodology (JPSM) at the University of 
Maryland, a program set up with U.S. government funds to 
address the shortage of trained survey researchers in the 
federal government. The JPSM is built on a collaboration 
of two universities (the University of Maryland and the 
University of Michigan) and a private survey research 
organization (Westat), with important contributions from 
experts in survey methodology in the government, other 
organizations, and other universities to support its various 
graduate programs. In a related approach, the Department 
of Social Statistics at the University of Southampton and 
the U.K. Office for National Statistics have recently jointly 
developed a master’s degree program in official statistics, 
with significant teaching contributions in both survey 
methodology and other aspects of official statistics being 
made by government statisticians. The Department is also 
collaborating with an independent survey research organi- 
zation (the National Centre for Social Research) in the 
Centre for Applied Social Surveys, one activity of which is 
to run short courses in survey methodology. 


3. DEVELOPMENTS IN SURVEY METHODS 


The computer revolution that began to have a significant 
impact on survey analysis in the 1960’s has been the domi- 
nating force behind the advancement of survey methodol- 
ogy over the past 25 to 30 years. The ability to process and 
analyze survey data much more readily than in the past has 
supported the use of more advanced statistical methods. It 
has also contributed greatly to more sophisticated demands 
from survey data users, stimulating the development of 
improved methodology for all aspects of the survey process. 

The chapter on processing survey data in Survey 
Methods contains a description of punch cards that were 
widely used 30 years ago for the analysis of survey data, 
together with a description of unit record equipment 
(counter-sorters and tabulators) and computers. At that time 
computers were well on the way to replacing unit record 
equipment, but they were not routinely available to survey 
researchers. The computers of the day were large main- 
frame machines and punch cards were the usual input 
medium for survey data. Programs for survey analysis were 
limited in number and in scope. Today, the situation is, of 
course, totally different, and the impact of this change on 
survey research is hard to overstate. 

It is against this backdrop of the computing explosion 
that the advances in other aspects of survey methodology 
should be assessed. The rest of this section briefly outlines 
what I view to be the significant advances that have been 
made in the past quarter century in the areas of question- 
naire design, data collection, missing data, survey sampling, 
and total survey error. 


Questionnaire design. The critical role of questionnaire 
design in achieving high-quality survey data has been well 


recognized from the early days. While some first-rate 
research was being conducted on improving questionnaire 
design in the 1960’s and 1970’s, the number of researchers 
involved in tackling this extremely challenging area was 
very limited. This situation has improved subsequently in 
large part due to what has become known as the Cognitive 
Aspects of Survey Methodology (CASM) movement. The 
CASM movement aims to attract researchers from the 
cognitive and social sciences to address the difficult 
problems of formulating survey questions that produce 
appropriate responses. The attention generated by this 
movement has created renewed interest in this field. 

The CASM movement has not identified ready-made 
solutions to the problems of response errors in surveys. It 
would have been unrealistic to expect that all that was 
needed was the importation of existing theories from 
cognitive psychology and other disciplines into question- 
naire design. What the movement has achieved is greater 
efforts to tackle the subject from a theoretical perspective. 
Also, the CASM movement has contributed greatly to more 
rigorous pretesting of survey questionnaires. Some of the 
pretesting techniques that have been developed in the past 
25 years occurred independently of the CASM movement, 
but the sustained attention that pretesting now receives 
owes a great deal to that movement. A direct effect of the 
CASM movement has been the creation of the so-called 
“cognitive laboratories” that are now widely used for 
pretesting questionnaires, using such techniques as “think 
alouds” and extensive probing. Focus groups — which have 
a long history in questionnaire design, particularly in 
market research — are also much more widely used than in 
the past. In addition, behavior coding is now used widely 
in pretesting. 

An associated development in the past few years has 
been a more theoretical approach to the design of forms that 
are to be completed by survey respondents. This research 
takes account of theories that indicate how individuals 
approach documents and how they most naturally work 
their way through them. This important subject received 
little attention for many years. The current research holds 
considerable promise for making survey forms much more 
user friendly, with the hope that this may improve both the 
quality of the data collected and response rates. 


Data collection. Survey Methods contains two main 
chapters on data collection methods, one on mail question- 
naires and one on face-to-face interviewing (there is also a 
chapter on documents and observation). There are only a 
few minor references to telephone interviewing, in part 
because of the low level of telephone penetration in the 
United Kingdom at that time. However, even in the United 
States where telephone penetration was much higher, back 
in 1975 many survey researchers had serious doubts about 
the collection of data for household surveys by telephone, 
at least for government surveys with major policy implica- 
tions. That situation has changed considerably. Today, 
many U.S. government surveys are conducted by telephone. 
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One concern about telephone surveys is the noncoverage 
of households without telephones. With telephone cover- 
age in the U.S. currently around 95 percent, the noncover- 
age of nontelephone households may be considered accept- 
able for surveys of the general population. However, a siz- 
able number of surveys focus on subpopulations with lower 
telephone coverage rates, such as the poor; for such sur- 
veys telephone noncoverage is a serious concern. Another 
concern is nonresponse. Nonresponse rates for telephone 
surveys are appreciably higher than for comparable face-to- 
face interview surveys, and the gap appears to be widening. 
In making a choice between telephone and face-to-face 
modes of data collection, the large cost savings that accrue 
from the use of telephone interviewing often override the 
higher response rates achievable with face-to-face inter- 
viewing. Nevertheless, the risk of appreciable bias that is 
associated with high levels of nonresponse in telephone 
surveys (frequently as high as 40 percent or more, even with 
determined follow-up efforts) is a serious and often under- 
rated concern. The likelihood of increasing nonresponse 
rates to telephone surveys raises questions about the role of 
telephone data collection in the future. 

An important advance in data collection methods in 
recent years has been the introduction of computer-assisted 
methods, such as computer-assisted personal interviewing 
(CAPI) and computer-assisted telephone interviewing 
(CATI. These methods facilitate more complex skip 
patterns, prevent interviewers from deviating from the 
specified question sequence, provide for easy insertion of 
responses from earlier questions (e.g., if a son’s name is 
recorded as “Peter” in answer to one question, “Peter” can 
be inserted in the wording of a subsequent question), and 
enable edit checks to be carried out as the interview 
progresses and corrections made as necessary. By entering 
the data directly into a computer file, they also permit more 
timely processing. The development of general purpose 
programs for CAPI or CATI data collections, including 
sampling and scheduling, is a complex operation. Several 
programs are now available for this purpose. Future devel- 
opments should see more flexible programs and authoring 
systems that are simpler to apply. 

In the past few years, another form of computer-assisted 
survey information collection has emerged. This is 
computer-assisted self-interviewing (CASJ), of which there 
are several variants: video-CASI, in which the respondent 
reads the questions on the computer screen and enters the 
answers on the keyboard; audio-CASI, in which the 
respondent listens to questions on headphones connected to 
a laptop computer and enters the answers on a keyboard; 
and telephone audio-CASI in which the audio-CASI inter- 
view is conducted by telephone, either with the respondent 
calling into the computer or with the respondent being 
transferred to the computer interview once the call has been 
established by a telephone interviewer. All these versions 
of CASI avoid the respondent-interviewer interactions that 
apply with other interviewing methods, and may therefore 


be particularly useful for collecting data on sensitive issues. 
They can also be developed in different languages if neces- 
sary. The audio variants avoid the requirement that the 
respondent is literate. These methods have appeared only 
recently and their use may be expected to expand appre- 
ciably in the future. 

Some business surveys are now conducted using audio- 
CASI methods. An advantage to respondents is that they 
can call in to a toll-free number at a time convenient to 
them. They then listen to voice-digitized survey questions 
and enter responses on the keypad of a touchtone telephone. 
A variant of this methodology is for the respondents to an- 
swer verbally, with the responses interpreted using voice 
recognition techniques. The use of this methodology may 
increase in the future as voice recognition methods 
improve. 

Another recent development has been the collection of 
survey data over the Internet. This methodology is particu- 
larly attractive for some types of establishment surveys and 
for surveys of populations of individuals who have access 
to the Internet and experience in using it. One approach is 
to send the questionnaire by email, which may be suitable 
for individuals who have known email addresses (e.g., the 
employees of a firm with its own network). Another 
approach is to post the questionnaire at a web site, with 
respondents using a password to gain access to it. At this 
time, the Internet is not appropriate for use in surveys of the 
general population because of the high proportion of 
persons without ready access to it, the lack of a sampling 
frame, and likely low response rates. The temptation to 
collect a large sample of Internet responses to a survey 
questionnaire in an uncontrolled fashion should be avoided. 
This approach would simply replicate the errors made with 
the infamous 1936 Literary Digest Poll. 


Missing data. Missing data occur in surveys through total 
nonresponse, item nonresponse, and noncoverage. During 
the past 25 years and even earlier, there has been increasing 
concern that total nonresponse rates have been rising. This 
trend is hard to document and indeed analyses of trend data 
from different surveys have led to different conclusions 
about the existence of a trend. Yet there is common agree- 
ment among survey practitioners that it has become more 
difficult over time to obtain cooperation. Various reasons 
have been suggested, such as less novelty in participating in 
a survey, more working people with less leisure time, fear 
of crime in face-to-face surveys, and the negative effects of 
telemarketing in telephone surveys, but there are no defini- 
tive explanations. Whatever the reasons, greater efforts 
now need to be made to achieve a high response rate than 
was the case in earlier times. These efforts include 
increased numbers of calls to contact respondents, greater 
efforts in refusal conversion, and the greater use of incen- 
tives. In the past decade, a sizeable number of experimental 
studies have been conducted in face-to-face and telephone 
interview surveys to test the effects on response rates of 
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various monetary and nonmonetary incentives and the level 
of monetary incentives, thus replicating in an interview 
setting the kinds of studies that were conducted with mail 
questionnaires in earlier decades. 

Noncoverage is a recognized concern in telephone 
surveys, but it has received less attention in face-to-face 
interview surveys, and certainly less attention than the pro- 
blem of nonresponse. Yet the level of noncoverage in face- 
to-face interview surveys among certain segments of the 
population (e.g., young black males in the United States) can 
be high. Moreover, little is known about those not covered, 
except that they can be expected to be different in many ways 
from those covered. It is a source of survey error that would 
benefit from greater attention in the future. Noncoverage is 
often especially severe when a survey of a rare population 
(e.g., teenagers) 1s conducted with sample members being 
identified through a large-scale screening survey. Given the 
increasing interest in surveying rare populations, this type of 
noncoverage warrants particular attention. 

Twenty-five years ago, item nonresponse was generally 
handled by simply dropping the cases from the analysis in 
question, for example computing percentage distributions 
for the subset of cases with acceptable responses. In 
essence, the implicit assumption being made was that the 
item nonresponses were missing completely at random 
(MCAR). Although that practice is still applied in many 
surveys, increasingly some form of imputation is being used 
to assign values for the missing responses in a manner that 
takes account of responses to other survey questions. This 
process replaces the often untenable MCAR assumption by 
a missing at random (MAR) assumption, that is that the 
item nonresponses are missing at random conditional on the 
auxiliary variables used in the imputation. Although impu- 
tation methods were occasionally used 25 years ago, most 
of the substantial literature on the subject has appeared 
since 1975. Current methods rely heavily on the computer 
power that is now available. Imputation remains an area of 
active research with two main foci: the development of 
imputation methods that maintain the covariance structure 
of the survey data set, taking into account that nearly all of 
the survey variables may be subject to item nonresponse, 
and the computation of variance estimates for survey esti- 
mates that are based on data some of which are imputed 
(see the discussion below). 

Data editing is closely related to imputation. It has also 
experienced significant advances in recent years, taking 
advantage of increased computing power to develop more 
complex editing procedures than could have been employed 
in the past. Like imputation, editing is the subject of much 
current research interest and further developments can be 
expected. 

The growth in computing power is also a major factor in 
the development and widespread use of weighting adjust- 
ments for nonresponse and noncoverage. Weighting class 
adjustments for nonresponse and noncoverage (poststratifi- 
cation) were applied when unit record equipment was used 


rf 


for survey analysis, but the methods were necessarily 
relatively simple. Now, more complex weighting class 
methods and calibration methods incorporating numerous 
auxiliary variables are widely used, often after exploratory 
analyses have been conducted to identify appropriate 
auxiliary variables. 


Survey sampling. The main methods of sample design 
(e.g., Stratification, multistage sampling, sampling with 
unequal probabilities) were developed in the early years and 
were described in textbooks that appeared in the 1950’s. 
The developments in the past quarter century have been 
refinements and extensions of these methods, for example 
to random digit dialing (RDD) sampling for telephone 
surveys. Here again, the ability of the computer to process 
large volumes of data in census files and other large 
sampling frames has enabled survey statisticians to con- 
struct more efficient sample designs than in the past. 

One area of research in recent years has been on methods 
for sampling rare populations, either in a special survey or 
by oversampling in a general survey. This interest is part of 
the extension of survey demands to provide results for many 
different domains, including small domains such as racial 
and ethnic minorities, children in poverty, age/sex groups, 
and geographical subdivisions (see also the reference to 
small area estimation below). The aim of the research is to 
develop efficient sample designs and data collection 
methods for sampling such domains in situations where 
special frames for those domains are unavailable. Since the 
demands for domain results continue to grow, ways to 
survey rare populations in a cost-effective manner will 
continue to be sought. 

In the 1970’s, the design-based mode of inference that is 
generally adopted with sample surveys was strongly chal- 
lenged by those who argued that it should be replaced by 
the model-dependent methods used in the rest of statistics. 
That debate has waned, and the design-based framework 
remains in place (see the further discussion below). In this 
context, the terminology should be clarified: from early on, 
the design-based mode of inference incorporated the use of 
models in improving the precision of survey estimates (e.g., 
regression estimates), but the estimates remained consistent 
under that mode of inference irrespective of the validity of 
the model. Thus, the procedures are model-assisted as 
distinct from model-dependent. The suitability of model- 
dependent estimates depends on the validity of the model 
(or the robustness of the estimates to model failure). The 
computing developments of recent years have facilitated the 
greater use of models, and of more complex models, within 
the design-based model-assisted framework of inference. 

These remarks should not be interpreted to imply that 
model-dependent methods have no place in survey research. 
On the contrary, the methods for handling missing data 
described above are necessarily model-dependent. Model- 
dependent methods are also used increasingly in producing 
estimates for small domains (generally small geographic 
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areas). Such methods are needed when the sample sizes in 
the domains are too small (they may often be zero) to 
produce design-based estimates of adequate precision. In 
this situation, small area estimates may be produced by 
borrowing strength from survey data for other areas or time 
periods through a statistical model that relates the survey 
data to other, generally administrative, data. The rapid 
growth in social programs that distribute funds to small 
geographic entities has led to a substantial demand for up- 
to-date small area estimates. As a result, small area esti- 
mation has become a major area of research activity in 
recent years, and is likely to remain so in the years to come. 

Variance estimation for estimates from complex sample 
designs has been another major area of development in the 
past quarter century. Methods based on Taylor’s series 
approximations and replication methods were being used in 
the 1960’s, but they were not routinely applied and were 
largely confined to research studies. This situation has 
changed dramatically as a result of the increases in com- 
puting power and the development of a number of computer 
packages for the computation of sampling errors for 
estimates from complex (typically stratified multistage) 
sample designs. It is now fairly common practice to com- 
pute sampling errors routinely in analyzing survey data. 

A notable development in recent years has occurred in 
the area of the application of analytic models to survey data. 
This area is one where there remains a debate about the 
choice between a design-based and model-dependent mode 
of inference. Within the design-based framework, there 
have been both theoretical advances in the application of 
regression models, categorical models, survival models, 
multilevel models, efc., with survey data and in software for 
computing variances for these models. At present, survey 
analysts often conduct their exploratory analyses using the 
greater flexibility of standard statistical packages, and com- 
pute the design-based variances using survey sampling 
variance estimation software only at the final stages of their 
analyses. In the future, survey sampling variance estima- 
tion procedures should become more fully integrated into 
standard packages. 

An area of much current research activity is the compu- 
tation of variance estimates for survey estimates that are 
based on responses some of which are imputed. One 
approach is the application of multiple imputation proce- 
dures to complex sample designs, an application that makes 
strong use of current computing power. Other methods are 
being developed under the standard design-based mode of 
inference (necessarily with model assumptions). The future 
may see the incorporation of these methods into the survey 
sampling variance estimation programs so that they can be 
readily applied. 


Total survey error. The preceding discussion has treated 
the various components of the survey process individually. 
A well-designed survey, however, is the blending together 
of the components into an effective package taking cost 
considerations into account. The last 25 years have seen a 


firmer recognition of the issue, with heightened attention to 
the concepts of total survey error and total survey design. 
With constrained resources, a survey design reflects trade- 
offs between, for example, sample size, the extent of non- 
response conversion undertaken, questionnaire length, and 
the quality of data obtained by different modes of data 
collection. In analyzing survey data, the quality of the 
estimates should properly be assessed in terms of the total 
survey error from all sources, not just sampling error. For 
both design and analysis, detailed information is needed on 
the various sources of error and their effects on the survey 
estimates. Moreover, since surveys are multipurpose 
studies, with many different analytic goals, the information 
requirements are extensive. The rapidly growing literature 
on survey errors from different sources is helpful for 
addressing total survey error and total survey design within 
cost constraints, but more studies are still needed. 

The total survey error and total survey design concepts 
are most readily applied to repeated surveys. Information 
on error sources can be accumulated from one round to the 
next and can then be used to determine priorities for where 
improvements in the survey methods are most needed. One 
use of the quality profiles that provide integrated accounts 
of what is known about the error sources in a survey (see 
the discussion below) is to guide the choice of priorities for 
methodological improvements. 


4. OTHER DEVELOPMENTS 


This section reviews a number of areas of survey 
research in which important developments have occurred in 
the past 25 years, other than the strictly methodological 
areas discussed in section 3. The set is not intended to be 
an exhaustive one. It includes only areas that I consider to 
have undergone major change. 


Panel surveys. The benefits of longitudinal data obtained 
from panel surveys have long been recognized, and panel 
surveys were being conducted in the 1940’s and 1950’s. At 
that time, however, the complexities of creating longitudinal 
data sets, combining the data collected in different waves, 
were severe. Panel surveys were often mostly analyzed only 
cross-sectionally, and this was a major source of criticism of 
the method. Today, the advances in computing and also in 
techniques for longitudinal analysis have changed the situa- 
tion dramatically. Nevertheless, the complexities of longitu- 
dina] data, especially the problem of missing data, remain. 
Longitudinal methods of analysis are now widely used, 
although many panel surveys are still analyzed mostly cross-- 
sectionally, with too little attention to the wide range of 
issues that their longitudinal data could illuminate. 

There has been an enormous growth in panel surveys in 
the past 20 years, covering a wide range of subjects, 
including education, labor force transitions, health, and 
voting behavior. Panel surveys of household economics, 
modeled on the University of Michigan's Panel Study of 
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Income Dynamics that began in 1968, have become popular 
and are now being conducted in a sizeable number of 
countries. There are also panels like Statistics Canada’s 
Survey of Labour and Income Dynamics and the U.S. 
Census Bureau’s Survey of Income and Program 
Participation that use similar approaches. 

It seems likely that the use of panel designs will increase 
even more in the future. The challenge is to make full use 
of the longitudinal data produced, since the analytic poten- 
tial of a panel survey increases exponentially with the 
number of waves of data it collects. In addition, the signifi- 
cant advances in techniques for longitudinal analysis being 
made by biostatisticians and others provide the tools for 
more sophisticated analyses than in the past. Many skilled 
analysts are needed if the data collected in a panel survey 
are to be fully analyzed. The growth of secondary analysis 
(see the discussion below) holds promise for fuller use of 
panel survey data in the future. 


International surveys. The last 25 years have seen the 
emergence of international surveys of various kinds, 
ranging from surveys promoted by international agencies to 
the coordination of independent country surveys to provide 
cross-national comparisons. A major breakthrough in this 
area came with the World Fertility Survey (WFS), which 
conducted surveys in 42 developing countries and 20 
developed countries during the period 1974-1982. The 
WES not only collected valuable data on fertility, but in 
many countries it also provided technical assistance in 
survey research that helped to develop an infrastructure of 
survey taking. The ongoing Demographic and Health 
Survey began shortly after the end of the WFS and to date 
has conducted surveys in more than 50 countries. 

Education has been the subject of a number of interna- 
tional surveys including, for example, the Third Inter- 
national Mathematics and Science Study (41 countries in 
1995) and its replication (40 countries in 1999); the 
Programme for International Student Assessment (about 30 
countries in 2000); the Second Civics in Education Study 
(about 20 countries in 1999); the IEA Reading Literacy 
Study (about 30 countries in 1991). The ongoing 
International Adult Reading Literacy Survey is collecting 
comparable information about literacy levels of adults in a 
number of countries around the world. Two examples of 
other internationally organized survey designs are the 
Multiple Indicator Cluster Survey from UNICEF and the 
Social Dimensions of Adjustment Integrated Survey from 
the World Bank. A related activity is the coordination of 
surveys in the European Union by Eurostat. An example of 
cross-national collaboration on surveys is provided by the 
International Social Survey Programme, a continuing 
annual survey program on social science topics that now 
has 33 member countries. 

The development of international survey programs has 
occurred for two separate reasons. One is the growing 
interest in the comparison of survey results across countries. 


The other is to assist countries, particularly developing and 
transition countries with limited survey experience, in the 
conduct of surveys that will provide important data for plan- 
ning purposes. Considerable expansion in international 
survey activity can be expected in the future for both of 
these reasons. 


Linkages to administrative data. The increases in 
computing power and the resultant ability to conduct more 
sophisticated analyses have led to a demand for more data 
on the sampled units. Analysts want to answer more 
complex questions than was the case in the past and some 
of the data they need may not be readily collectable in a 
survey, at least with the required level of quality. Even if 
the data were collectable, the collection could create exces- 
sive respondent burden. This situation has led to the search 
for alternative sources for the data, with data taken from 
those sources then being linked to the survey responses. 
Thus, for example, tax records might provide valuable 
earnings histories for sampled individuals over a timespan 
for which the respondents could not provide the data, or 
medical records might provide the amounts of medical 
expenses paid directly by insurers that are unknown to the 
respondents. These kinds of linkages have been made 
much more feasible by the significant expansion in the 
number of administrative record systems now available in 
electronic form. 

There has been considerable interest in linking admini- 
strative record data to social survey data in recent years and 
a number of surveys have made such linkages. However, 
there are generally significant problems to overcome in 
gaining access to administrative data and serious concerns 
about protecting the survey respondents’ privacy. These 
issues have severely limited the use of administrative record 
linkages in household surveys to date. Despite the substan- 
tial potential benefits of such linkages, it is not clear to what 
extent these barriers can be overcome. 

In contrast, administrative data have become a key 
element in the conduct of economic surveys and censuses 
and, in a number of cases, they have replaced the data that 
used to be collected from respondents. The result has been 
a substantial decrease in respondent burden, improved data 
quality, more timely reporting, and reduced costs. 


Secondary analysis. The increases in computing power, 
the increasing numbers of surveys being conducted, and the 
increased sophistication of the data collected in surveys 
have all stimulated a major growth in the secondary analysis 
of survey data. Public-use files are now more routinely 
made available, sometimes through survey data archives, to 
enable secondary analysts to conduct their own analyses, 
thus permitting survey data to be more thoroughly analyzed. 
Associated with this activity, increased attention has been 
needed to protect the survey respondents’ confidentiality 
and to ensure that data files released to secondary analysts 
are not used to breach confidentiality. With secondary 
analysis undoubtedly continuing to expand in the future, 
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continued attention will need to be given to ways to release 
survey data in a manner that protects respondents but does 
not seriously curtail the range of analyses that can be 
conducted. 


Survey quality. Increasing attention is being given to 
different aspects of survey quality. In the past few years, a 
number of survey organizations have become interested in 
survey process quality, applying the ideas of total quality 
management to survey processes. Greater attention than in 
the past is being given to quality taken in the broad sense to 
include the accuracy of the estimates produced, relevance, 
timeliness, accessibility and cost-efficiency and in the 
narrower sense of accuracy alone. Users of survey esti- 
mates and secondary analysts of survey data need to be 
informed about the overall quality of the survey data, 
including sampling errors, nonresponse and noncoverage, 
response errors and processing errors. While this need has 
long been recognized, current practice in reporting survey 
quality is often seriously deficient. There are signs that 
more attention is now being given to this area. The 
introduction of quality profiles that provide full and inte- 
grated reports on the quality of the data in ongoing surveys 
is an important contribution. 


5. CONCLUDING REMARKS 


This section attempts to predict some major considera- 
tions for survey research in the next 10 to 20 years. The 
computer revolution that has transformed the nature of 
survey research over the past 25 years is still in progress, 
and further developments can be expected in many aspects 
of collecting, processing, and analyzing survey data. The 
telecommunications industry is also in a state of rapid inno- 
vation, and the changes are likely to affect the ways that 
survey data are collected. It seems likely that greater use 
will be made in the future of mixed-mode designs, taking 
advantage of new modes for respondents with access to 
them (e.g., the Internet) and using conventional modes for 
other respondents. Thus the effect of mode on survey 
responses will continue to be an important concern. 

In general, it seems probable that the demand for survey 
data will continue to grow rapidly as more policy analysts 
learn to take advantage of survey data. Increasingly, survey 
estimates will be needed for small domains, especially small 
geographic domains, as policymakers target their programs 


to special population subgroups. Currently, most of the 
demand for survey data comes from central governments; 
in the future the demand from provincial and local govern- 
ments may expand. The difficulty here is that surveys cost 
almost as much for small populations as for large ones. 
Local governments may therefore often be unable to afford 
the cost of a survey unless inexpensive methods can be 
found. 

The major concern for the future of survey research is 
that respondents’ willingness to participate in surveys may 
continue to decline, and that increased efforts in data collec- 
tion will not fully counteract this effect. Thus, response 
rates will fall. This comment is of particular salience for 
telephone surveys, where nonresponse rates are already 
high. A significant increase in telephone nonresponse rates 
could even lead to the demise of telephone data collection 
for household surveys. 

Finally, the next decade or so may well see the 
emergence of a new and different professional society for 
survey researchers that more broadly represents the interests 
of all members of the profession. Since survey sampling 
was at the forefront of the developments of survey research 
in the early years, survey research has strong ties with 
statistical societies. However, those ties tend to concentrate 
on survey statistics. There are also ties with societies for 
public opinion research, market research, and various sub- 
ject matter disciplines, such as sociology and psychology, 
primarily for survey researchers who deal with the 
nonsampling aspects of survey research. Similarly, there 
are ties with computing societies for those working on 
survey computing. As yet, however, there is no society that 
aims to bring survey researchers of all disciplines together. 
The years to come may see the creation of such a society to 
promote exchanges across the different disciplines and 
thereby help to advance the field. Were such a society to be 
formed, it would not affect the need for the current ties that 
survey researchers have with statistical and other societies. 
Survey researchers need to keep in touch both with the 
developments taking place in survey research broadly and 
also with the developments in their own disciplines. 
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Survey Sampling Theory Over the Twentieth Century 
and its Relation to Computing Technology 


D.R. BELLHOUSE! 


ABSTRACT 


Computation is an integral part of statistical analysis in general and survey sampling in particular. What kinds of analyses 
can be carried out will depend upon what kind of computational power is available. The general development of sampling 
theory is traced in connection with technological developments in computation. What is possible in theory is only 
practicable with the proper computing technology. At the same time new developments in technology can motivate new 
areas of theory to investigate. One hundred years ago, it was the requirements of statisticians that spurred on technological 
development. Although theoretical developments in sampling theory have often run ahead of computational capabilities, 
it is now the case that survey statisticians are now followers of computing technology that has been motivated by others 
instead of acting as the catalyst that leads to technological change. 


KEY WORDS: Analysis of survey data; Digital computers; Punch cards; Scientific programming; Statistical software; 


Survey data analysis; Survey estimation. 


1. INTRODUCTION 


There are several ways to approach the history of survey 
sampling. Two are very tempting, but will not be followed 
here. The first is to examine sampling in the context of the 
history of ideas -- who formulated them and then how and 
why they are formulated, promoted, defended and discarded 
or supplanted. With respect to the personalities, it is not 
necessarily the one who espouses the idea first who is given 
prominence but the one who promotes it the best or the one 
who can best put the idea into practice. The approach of the 
history of ideas has been followed to a certain extent by 
Kruskal and Mosteller (1980) and Bellhouse (1988) who 
examined the progression of ideas beginning with the 
espousal of the representative method by Kaier (1897) over 
censuses combined with the use of randomization in 
surveys by Bowley (1906). The whole story of the debates 
over the foundations of sampling falls directly under this 
approach. From this debate, which was initiated by 
Godambe (1955), has emerged the continuing question of 
when to use models in sampling design and estimation. A 
second way to approach the history of sampling is to look 
at sampling theory as a branch of mathematics and then to 
fit this development into the general pattern of how research 
in mathematics evolves. Complicating this is that there are 
several approaches to how mathematics evolves, as discus- 
sed in Gillies (1992). One approach is to note that periodi- 
cally there are results which seem to open up new areas of 
research while other areas become seemingly complete or 
“fished out” for new research ideas. Emerging areas of 
research often attract several talented researchers to work 
on these new problems and away from other potential 
research problems. This has its parallels in sampling. 
Hansen and Hurwitz (1943) obtained results on sampling 


with probability proportional to size and with replacement. 
Then Horvitz and Thompson (1952) extended this idea to 
sampling without replacement. The basic problem in 
unequal probability sampling without replacement is to find 
a sampling design that yields the desired inclusion probabil- 
ities. This resulted in several papers on the subject culmina- 
ting in the review monograph by Brewer and Hanif (1983). 
Lately, very few papers are written to promote new without 
replacement sampling designs that result in inclusion proba- 
bilities proportional to a size variable. However, statistics 
and survey sampling cannot be equated to pure mathemat- 
ics. Much of statistical research is motivated by practical 
problems in data interpretation and analysis not by abstract 
ideas. 

In view of the explosion of technology over the 20th 
century, I chose another approach. This is to view the 
history of sampling over the 20th century as the history of 
the interplay between ideas that have been put into practice 
and computing technology that has defined the limits of 
practice or that has encouraged ideas for new developments 
in practice. The development of sampling methods may be 
categorized by the intersection of two strands: the use of 
surveys for descriptive and analytic purposes, and whether 
or not hypothetical models should be used. 


2. BEGINNINGS: THE FIRST HALF OF THE 
TWENTIETH CENTURY 


The first two major breakthroughs for survey sampling, 
one in the formulation of a statistical concept and the other 
in the development of technology, occurred at the end of the 
nineteenth century. Both breakthroughs faced some initial 
opposition or apathy, the idea more so than the technology, 
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but both prevailed and were developed further. These 
breakthroughs were: (1) Kaier’s (1895/6, 1897, 1905) 
espousal of sampling through a “representative method” 
over attempts at complete enumeration for social surveys, 
and (2) the development of punch card machines for data 
processing by Hollerith (1894). Both breakthroughs were 
directly related to survey or census work. This was the first 
and last time that survey or census issues inspired major 
technological innovation. From then on, survey sampling 
has adapted itself to the available technology. 

Kaier’s idea was to get a sample that was an approximate 
miniature of the population. Through sampling, more de- 
tailed information could be obtained and more specialized 
studies could be carried out, all at a fraction of the cost of 
a census. The idea initially met with opposition and it took 
upwards of a decade for his ideas to be accepted. 

The development of machinery by Herman Hollerith for 
data processing came directly out of the needs of the U.S. 
Bureau of the Census and the encouragement of the 
Bureau’s Director of Vital Statistics, John Shaw Billings. 
The events that led to this development are described by 
Willcox (1926): 


‘While the returns of the Tenth Census [1880] were 
being tabulated at Washington, Billings was 
walking with a companion through the office in 
which hundreds of clerks were engaged in labori- 
ously transferring items of information from 
schedules to the record sheets by the slow and heart- 
breaking method of hand tallying. As they were 
watching the clerks he said to his companion, 
“There ought to be some mechanical way of doing 
this job, something on the principle of the Jacquard 
loom, perhaps, whereby holes on a card regulate the 
pattern to be woven.’ The seed fell on good ground. 
His companion was a young talented engineer in the 
office who first convinced himself that the idea was 
practicable and then that Billings had no desire to 
claim or use it. Thereafter he devoted the bulk of his 
life with great ultimate profit for himself and the 
world to ripening the invention and securing its 
adoption. I have no need to describe or eulogize 
Hollerith machines.” 


A full description of the development and use of these 
machines for surveys is given in Mandeville (1946). 
Hollerith’s machine was applied to processing the 1890 
U.S. census. While the 1880 census took over seven years 
to complete, the 1890 census was finished by early 1895. 
The Bureau used 180 tons of cards that were processed at 
a speed of 6,900 cards per 62-hour day. Not only did the 
machine save time, it also significantly reduced tabulation 
errors. The punched card machine was used to process the 
189] Census of Canada, but it did not see early use in the 
censuses of the United Kingdom and the rest of the British 
Empire. It was felt that the level of detail required in these 


censuses did not justify the use of a Hollerith machine since 
the time saved by the machine would be balanced by the 
time taken to punch the card (Hooker 1894). In a paper on 
census taking Baines (1900) expressed a preference for 
manual over machine tabulation, especially when labour 
was cheap. Despite these initial misgivings, improvements 
to the machine continued and the use of the Hollerith 
machine for statistics became highly developed by mid- 
century. Hartley (1946) demonstrated the most sophisti- 
cated use of these punched card machines for statistical 
analysis. This included the calculation of moving averages 
and serial correlations as well as the solution of simulta- 
neous equations on Hollerith machines. 

After these near simultaneous and unrelated innovations 
in ideas and technology, theory ran ahead of practice for the 
next 50 or 60 years. Theoretical developments in sampling 
continued through the first half of the century. Out of dis- 
cussions over the path to follow in the “representative 
method,” Bowley (1926) put together a monograph 
describing all the known theoretical results in sampling 
under random selection and under purposive selection. In 
addition, he developed the theory for stratified sampling 
under proportional allocation. The triumph of randomiza- 
tion over purposive selection was due to Neyman (1934) 
who showed why randomization gave a more reasonable 
solution to sampling problems than purposive selection. 
Although not the first to do so, he also developed optimal 
allocation strategies for stratified sampling. Prior to the 
middle of the century the last major development, in terms 
of sampling design with accompanying estimates and vari- 
ance estimates, was the concept of unequal probability 
sampling introduced by Hansen and Hurwitz (1943). 

The practical implementation of these theoretical results 
was limited to relatively small-scale surveys. The analyses 
for most surveys used calculators, either electric ones such 
as those manufactured by Friden, Marchant or Monroe, or 
hand calculators operated by turning a crank such as the 
Brunsviga used by Pearson and the Millionaire used by 
Fisher. Since the labour in the analysis increased signifi- 
cantly with the sample size, standard errors were seldom 
calculated, and when calculated the correct formulas were 
seldom applied. Bowley (1936) describes a typical situation 
showing the infrequency of standard error calculations: 


“Tabulation is usually a dull and tedious job, but 
there is a certain interest in watching the entries ac- 
cumulating in a cross table and seeing the gradual 
growth of continuity out of randomness. When the 
results take the form of a frequency curve, and espe- 
cially if we have reason to expect a normal curve 
and find it, we have good reason to suppose that we 
have measured satisfactorily a real entity. Thus the 
distribution of price changes or their logarithms on 
a normal scale gives a great deal of support to the 
validity of an index number. In such cases the 
computation of standard error is reasonable.” 
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Box and Thomas (1944) describe a survey of approximately 
4,500 respondents stratified by the industry in which they 
worked. The standard errors, when presented, were calcu- 
lated using the formula for simple random sampling. A 
decade later Deming (1956) noted: 


“Although the possibility of showing a valid stan- 
dard error is by definition a feature of any probability 
sample, it is a fact that results of probability samples 
have too often appeared in the past without standard 
errors because of the sheer labor of computation.” 


It is within this context that Mahalanobis (1946) sug- 
gested the technique of interpenetrating subsamples. This 
technique, which Mahalanobis developed at the Indian 
Statistical Institute in the 1930’s (Murthy 1967 and Deming 
1956), is very simple: two or more independent subsamples 
are chosen according to the same sampling design. Then the 
variation between the subsample estimates of the population 
total provides an unbiased estimate of the variance of the 
final estimator of the total. Computationally, the method has 
distinct advantages in the punch card environment where 
sums are easier to obtain than variances. With interpene- 
trating subsamples the main computational effort is in 
finding the subsample estimates that are based on sums 
only. The Indian Statistical Institute obtained its first 
Hollerith machine in 1944. Prior to that time, tabulations 
and other calculations were done by hand. The Institute’s 
Annual Report for 1945-46 published in Sankhya shows the 
initial unease that always greets technological change and 
the eventual positive benefits to change. With respect to the 
introduction of these machines, the report states: 


“Contrary to apprehensions among certain sections 
of workers that the Hollerith machine would to a 
large extent eliminate manual computations, it was 
found that new and detailed studies which could not 
be formerly undertaken could now be handled with- 
out difficulty so that the demand for trained com- 
puters in the later stages was on the increase. In 
addition to routine projects undertaken from time to 
time, special studies such as mechanical solution of 
determinants, construction of tables, fitting of ortho- 
gonal polynomials, etc. were conducted.” 


In the United States, Deming (1956), for example, picked 
up on the general idea and put forward methods of repli- 
cated sampling. The U.S. Bureau of the Census used this 
method for variance estimation. At the Bureau this idea 
evolved into pseudo-replication, or eventually balanced re- 
peated replication, for variance estimation (McCarthy 
1969). 
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3. THE ADVENT OF THE DIGITAL 
COMPUTER 


The initial development of the digital computer was for 
military purposes during the Second World War (Ceruzzi 
1998). For some years after the war the military continued 
to play a central role in the advancement of computing. By 
the 1950’s commercial uses were developed for the 
computer, and this is where sampling practice begins to 
catch up with sampling theory. The first generation of com- 
mercial computers included the UNIVAC followed by the 
IBM 700 series. These computers contained thousands of 
vacuum tubes as internal memory. The tubes for the IBM 
machine were about three inches in diameter and held 1,024 
bits of information. The UNIVAC ran at 2.25 MHz and 
could carry out 465 multiplications per second. For both 
machines, data were input via punched cards and stored 
data was on magnetic tape rather than continued use of the 
punched cards. The 1961 census in the United Kingdom 
underscores the continuing central role of the military in 
computing at this point in time. The census was processed 
on an IBM 705 computer (Benjamin 1961). The computer 
belonged to the War Office and was used by the Royal 
Army Pay Corps. The census workers were able to use the 
computer when not in use by the army. Information was 
input via cards punched in one location and then taken to 
the computer in another location. 

Although it was not at the forefront of the development 
of the computer as it had been with the Hollerith equipment, 
the U.S. Bureau of the Census was central in the initial 
commercial development of the digital computer. Not only 
did the Bureau receive the first UNIVAC that was pro- 
duced, but also some of its employees participated in design 
decisions for its construction (Ceruzzi 1998 and Hansen 
1987). The computer was delivered in March of 1951 and 
was used for processing the 1950 census. It ran 24 hours a 
day all week until the task was completed. Once the census 
work was completed, the computer was used for other 
censuses and surveys including the Current Population 
Survey. Technology was now catching up to theory; the 
computer was now used for better calculation of variance 
estimates. It also opened up new possibilities, in particular 
imputation of missing values. With respect to variance 
estimates Hansen, Hurwitz, Nisselson and Sternberg (1955) 
comment: 


“Until the acquisition of a high-speed electronic 
computer, the UNIVAC, extensive approximations 
were introduced into the estimates of variances to 
avoid computations that would be exceedingly time 
consuming with the available equipment. The avail- 
ability of the UNIVAC makes it possible to avoid 
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most of these approximations. Even with the 
electronic computer, however, the work of making 
variance computations would be extremely heavy if 
variances were computed for all items directly. 
Approximate methods will continue to be used in 
the future, but they will be evaluated by more exact 
computations than have been feasible in the past.” 


Other statistical organizations followed but at a slower 
pace. The slow pace in Canada was perhaps due in part to 
the American experience. A 1956 report to the Dominion 
Statistician at the Dominion Bureau of Statistics in Canada 
on the subject of computing at the Bureau of the Census 
(reported and quoted by Worton 1998) states: 


“Subject-matter people ... are not entirely con- 
vinced that the UNIVAC system has given them the 
results which might be expected from a computer 
system. Undoubtedly UNIVAC has given a great 
deal of trouble — much of it probably not the fault of 
UNIVAC at all. Factors such as poor programming, 
inadequate analysis of the job, inexperienced oper- 
ating staff, maintenance problems, and even friction 
between the three operating groups, i.e., the subject 
matter staffs, the Central Operations Group, and the 
Central Electronics Unit are reflected in the perfor- 
mance of the UNIVAC system.” 


The Dominion Bureau of Statistics, now Statistics Canada, 
obtained its first computer in 1960, an IBM 705. The com- 
puter was used to process the 1961 census. As noted 
already, the British used an army-owned computer to pro- 
cess their 1961 census. In the late 1940’s, Mahalanobis was 
on a list showing interest in obtaining one of the first 
UNIVACs (Ceruzzi 1998). However, the annual reports of 
the Indian Statistical Institute published in Sankhya show 
that the Institute did not obtain a computer until 1956 at 
which time it received an HEC-2M. 

Variance estimation for survey estimates of means, totals 
and proportions was now feasible for large-scale surveys. 
Widespread use of this technology now depended on two 
things — access to a computer, which was an expensive item 
to buy, and appropriate software to carry out the calcula- 
tions. 


4. SCIENTIFIC PROGRAMMING 


Certain kinds of research, and the application of these 
research results, are possible only with computing. These 
possibilities expand not only with the expansion in com- 
puting power, but also with easier access to the computer’s 
power through programming languages or packaged pro- 
grams. For several years the most popular scientific 
programming language was FORTRAN (FORmula 
TRANslation). This was introduced in 1957 by IBM for its 


704 computer. Part of what popularized FORTRAN was 
the development of the WATFOR (WATerloo FORtran) 
compiler at the University of Waterloo in 1965. This 
popular compiler, which was used for teaching purposes, 
combined with the dominance of IBM in the marketplace 
made FORTRAN accessible to many students and subse- 
quently to researchers (Ceruzzi 1998). In reporting on the 
development of his own computer programs for survey 
research, Yates (1973) shows how pervasive FORTRAN 
had become over the 1960’s. Yates’s programs for the 
computer at Rothamsted Experimental Station were origi- 
nally written in the late 1950’s with code specific to the 
computer they had. In the mid-1960’s the code was written 
in Extended Mercury Autocode. By the end of the 1960’s 
this code had to be translated into FORTRAN using a 
machine translator; otherwise it was not usable at any other 
computer location. The earliest use of FORTRAN in 
sampling that I can find is in Fan, Muller and Rezucha 
(1962). These three individuals, all of who worked at IBM, 
developed algorithms and accompanying FORTRAN code 
to select simple random samples by computer. 

There were two different paths that were followed in the 
application of FORTRAN programming to survey samp- 
ling. One was among statistical agencies or survey research 
centres and the other was among individual academic 
researchers. The kind of work followed along each path is 
strongly correlated with the evolving power of the computer 
and the dominance of IBM (and hence FORTRAN) in the 
market. By the end of the 1960’s, many institutions had new 
and more powerful mainframe computers, often one of the 
IBM 360 series that was originally announced in 1964. 
Moreover, the software (FORTRAN in particular) remained 
compatible with machine changes and upgrades, especially 
for machines in the IBM 360 series (Ceruzzi 1998). The 
Dominion Bureau of Statistics obtained its first IBM 360 in 
1969, while for example the Universities of Manitoba, 
Toronto and Waterloo obtained their first machines in the 
years 1966-67 (Day 1971). At the agencies and research 
centres, various formulae and procedures necessary to 
survey design and analysis were computerized. For 
example, Fellegi, Gray and Platek (1967) report that when 
the Canadian Labour Force Survey was redesigned over 
1964-65, sample selection by Fellegi’s (1963) method of 
unequal probability sampling was coded into a FORTRAN 
routine. From the University of Michigan Survey Research 
Center, Kish and Frankel (1970) report that they had 
FORTRAN code for obtaining variance estimates for a 
variety of statistics including regression coefficients using 
balanced repeated replication. By the mid-1960’s academic 
researchers began to use the computer via FORTRAN 
programming to study, numerically or empirically, the 
sampling theory that they or others had derived. One of the 
first was Sedransk (1965) who carried out some efficiency 
comparisons in FORTRAN on an IBM 7074 (marketed by 
IBM in 1964) for a double sampling scheme. In particular, 
efficiency comparisons were made between optimal values 
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for the first and second phase sample sizes and an approxi- 
mation to the optimal values. The computations involved 
taking expected values over a trinomial distribution in 
which several conditions had been imposed. The use of the 
computer here was to obtain a numerical comparison 
between exact methods and approximate ones. By the end 
of the decade a new kind of computer-based research 
process emerged. Rao and Bayless (1969) and Bayless and 
Rao (1970) compared several unequal probability sampling 
schemes by generating all possible samples and calculating 
the exact finite population mean square error for several 
real and constructed populations. It then became the norm 
to carry out extensive empirical studies on any newly pro- 
posed estimator or design. 

The past 30 years have seen remarkable changes in com- 
puting technology. Modern computers are much faster, 
physically smaller and have much greater storage capacity. 
The steady increase in computing power and the availability 
of standard programming languages has allowed survey 
researchers to expand as well into survey data analysis. This 
technological change is reflected in developments in samp- 
ling theory for variance estimation. From the 1960’s to the 
1980’s there were three basic computerized approaches to 
variance estimation of complex survey statistics: Taylor 
linearization (see Woodruff 1971, for early references to its 
usage), jackknife (first proposed in sampling by Durbin 
1959) and balanced repeated replication (McCarthy 1969). 
The rise of computing power saw a new technique, Efron’s 
(1982) bootstrap, for variance estimation. This new 
Statistical technique, which was contemporaneous with the 
development of networked RISC (Reduced Instruction Set 
Computing) workstations running under a UNIX operating 
system, is highly computer intensive. Over the 1980’s RISC 
workstations gradually replaced most mainframes in 
research organizations. Near the end of this transition away 
from mainframes, Rao and Wu (1987) extended bootstrap 
methodology to variance estimation for smooth statistics 
under stratified multistage designs. 

The most recent software to have an effect on statistical 
research is the development of computer algebra packages. 
Although computer algebra has been in existence for some 
time, it is only in the last decade that it has progressed to the 
point that it is accessible to many researchers. With com- 
puter algebra many complex manipulations can be done 
automatically and much quicker than by hand and without 
risk of error. Similar to several other areas of statistics, 
many of the algebraic manipulations in sampling theory are 
related to algorithms that generate partitions. Based on the 
computer algorithms developed by Andrews and Stafford 
(1993) and Stafford and Andrews (1993), Stafford and 
Bellhouse (1997) have extended computer algebra techni- 
ques to survey sampling theory. Using their methodology, 
most of the results of so-called classical sampling theory, 
either existing in the literature or yet to be obtained, can be 
derived automatically. 
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5. ANALYSIS OF SURVEY DATA 


While steady and substantial progress had been made in 
research on problems of survey estimation or enumerative 
surveys over the 20th century, by 1970 little had been 
accomplished on the analytical aspects of surveys. The 
terms “enumerative” and “analytical” surveys were coined 
by Deming in 1950 (Deming 1953). In the same article he 
also gives a succinct definition: 


“Briefly, the enumerative question is how many? 
The analytic question is why? is there a difference 
between two classes, and if so, how big are the 
differences?” 


There is an implication in this quotation that the purpose of 
analytical surveys was for comparisons of domain means. 
Certainly, throughout the 1960’s the understanding of what 
constituted an analytical survey was often limited to this. 
Cochran (1963) states: 


“In an analytical survey, comparisons are made bet- 
ween different subgroups of a population, in order 
to discover whether differences exist among them 
that may enable us to form or to verify hypotheses 
about the forces at work in the population.” 


Yates (1960) also focused mainly on domain comparisons 
in his discussion of analytic surveys. He did, however, 
discuss regression analysis and the problem of attenuation, 
but not the problem of general survey weights. Skinner, 
Holt and Smith (1989) attribute the pioneering work in 
analytical surveys to social scientists, Paul Lazarsfeld in 
particular. I will use the theoretical development of 
regression analysis in complex surveys to illustrate these 
connections to social science, in this case economics. 

One of the earliest studies to take into account the survey 
weights in regression analysis was by Klein and Morgan 
(1951). At the time both were at the University of 
Michigan; Morgan was in the Survey Research Center. At 
the outset of their paper they state: 


“The sample design, the methods of collecting the 
data, and underlying economic behavior will all 
contribute to the formulation of the model. The 
study of data collected in consumer surveys has 
convinced us that one cannot proceed simply by the 
application of conventional statistical methods in 
the estimation of economic relationships because of 
the existence of some basic difficulties which we 
classify as follows: (1) weighting of observations, 
(2) heteroscedasticity, (3) nonlinearities, (4) the 
choice of alternative economic concepts, (5) errors 
of observation.” 


They addressed the first four “‘basic difficulties” but not the 
fifth. In their analysis of the approximately 2,300 responses 
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to the Survey of Consumer Finances, which was a multi- 
stage sample, Klein and Morgan used the survey weights 
through weighted least squares estimation of the regression 
parameters but ignored the clustering effect when it came to 
variance estimation. They noted that in many cases the use 
of the survey weights had little effect on the estimates of the 
regression coefficient estimates but noted that there was a 
reduction in the estimated variance for the model error. 
Though Klein went elsewhere, Morgan remained at the 
Michigan Survey Research Center. Twenty years later, he 
and another (Lansing and Morgan 1971) gave an overview 
of the state of the art for the analysis of economic survey 
data. Not much had changed in terms of the incorporation 
of the survey design into the analysis. The same is true for 
other areas of social research; in many cases not even the 
survey weights were used. In the economics literature 
debate continued for at least twenty years over whether to 
use the survey weights in regression analysis; Porter (1973) 
has several references to this debate. 

It was out of this milieu that Kish, who also worked at 
the Michigan Survey Research Center, initially put forward 
the concept of the design effect (Kish 1957), which is the 
measure of increase or decrease in variance over simple 
random sampling experienced in a survey with a design 
other than simple random sampling. Design effects have 
become central to many aspects of the analysis of complex 
survey data. With respect to regression analysis, Kish and 
Frankel (1970) studied the design effects in the estimation 
of regression coefficients. They used balanced repeated 
replication to obtain their variance estimates. It is not 
entirely clear in their presentation exactly what regression 
coefficients they were estimating. Later, the parameters 
were explicitly spelled out in Kish and Frankel (1974). Spe- 
cifically, the finite population parameters are what would be 
obtained in least squares estimation of superpopulation 
regression parameters when the entire finite population is 
available. Estimation of these parameters has become one 
of the standard approaches to regression analysis from com- 
plex surveys. Fuller (1975), using Taylor approximations to 
the variances, put the whole inference process on a solid 
theoretical foundation by providing limit theorems for the 
estimates. In addition, he addressed the one problem that 
Klein and Morgan (1951) ignored: errors in the variables or 
measurement errors in the independent variables. 

Konijn (1962) took a different approach to regression 
analysis. Under a cluster sampling design, he assumed 
different simple linear regression models within each 
cluster. The parameters of interest were weighted averages 
of regression parameters with the weights given by the 
cluster sizes. This approach is model-based in the sense that 
it is the model parameters that are of interest, not a finite 
population parameter. Konijn’s approach was not followed 
for several years. However, there is now a substantial 
literature that has grown out of this model-based approach; 
Pfeffermann (1993) contains several references. 


With regard to the social science origins of survey 
analysis, there were similar experiences in categorical data 
analysis. The sociological literature from the 1960’s and on 
contains many examples of categorical data analysis 
ignoring the sampling design. After Rao and Scott (1981, 
1984) developed contingency table and goodness of fit 
analyses for complex surveys, Rao and Thomas (1988) tried 
to promote this methodology among sociologists using a 
review article. A search through citation indexes shows 
that, although this work has had great impact in the statisti- 
cal and medical literature, it has had little impact in the 
sociological literature. The reason for this may be due, in 
part, to lack of computer software. The most popular soft- 
ware among sociologists, which is SPSS, does not at the 
moment contain any routines for the analysis of complex 
survey data. This points to a wider problem: regression, 
categorical data analysis and other techniques that have 
been proposed for complex surveys are not widely practi- 
cable without the appropriate computer software. Fuller 
himself tried to respond to this need by developing a pack- 
aged program for survey data analysis (Hidiroglou, Fuller 
and Hickman 1980). 


6. STATISTICAL SOFTWARE FOR 
SURVEY RESEARCH 


Frank Yates at Rothamsted Experimental Station was the 
first statistician to develop software for survey research. His 
work began in the late 1950’s (Yates and Simpson 1960). 
Originally, programs were written that were specific to each 
survey. This evolved into a general-purpose program by the 
early 1960’s (Simpson 1961). Although it was the first in 
the field and was available for many years, it never 
achieved widespread popularity. There are at least four 
reasons for its general lack of success, reasons that point to 
the success of other software developers. 


(1) The package was not user friendly. In his obituary of 

Yates, Dyke (1995) made allusion to this fact. He 
says: 
“Yates believed that the analyst should 
understand the relevant theory, and so be ready 
to specify in exact detail what he wanted. 
Perhaps for this reason the program was not 
excessively easy to use! But its power and 
flexibility, and uncluttered clarity of its output 
were, and are, outstanding.” 


(2) It was too expensive for what it did and could not 
compete with cheaper competitors. Wolter (1985) lists 
a number of packages that were available in the 
mid-1980’s. At the time the package was twice as 
expensive as SUDAAN but could do only tabulations, 
whereas SUDAAN had the additional capability of 
regression analysis and ratio estimation. 
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(3) Marketing is an important factor in the success of a 
product. Yates appeared to be more interested in 
tinkering with his product to improve it rather than 
investing time in marketing it. 


(4) Other than a manual, by 1985 there was no technical 
support for the package. 


Yates was not alone in having software that did not catch 
on. I had the same experience when I developed variance 
estimation software based on tree traversal algorithms 
(Bellhouse 1985). Other than the expense factor (mine was 
free), my package was a living example for the other three 
reasons why some software does not fly. 

By the early 1970’s there were over 40 packaged 
programs and routines, written mainly in FORTRAN, that 
would do statistical analyses (Schucany, Minton and 
Shannon 1972). Of these original packages only two have 
remained popular in the marketplace, SAS first released in 
1970 and SPSS released in the late 1960’s. 

The survey software that has maintained predominance 
in the market for several years is SUDAAN developed by 
B.V. Shah of the Research Triangle Institute (Shah 1978 
and 1984). It is marketed well and fully supported by its 
developer. It was originally accessed as a SAS procedure 
and has now become a stand-alone package. The tie with 
SAS was probably one of the reasons for its initial success. 
Those who were familiar with SAS could easily familiarize 
themselves with this new procedure, or equivalently the 
package, so that in a sense it was user friendly. Further, the 
package has continued to keep pace with survey research. 
The original program contained routines to calculate stan- 
dard errors for survey estimates including means, totals, 
proportions and ratios. This was expanded to include 
regression analysis in the late 1970’s when research on 
regression in complex surveys was under way. The program 
now contains routines for regression analysis, logistic 
regression, categorical data analysis and survival analysis. 
It has also kept pace with developments in computing 
machinery. Originally developed on a mainframe computer, 
the package is now available for use on a PC. It still main- 
tains its links to SAS, although SAS currently has its own 
survey analysis procedures under development. 

Currently, there are several other programs for survey 
analysis. The most popular among these programs, in addi- 
tion to SUDAAN, are STATA and WesVarPC. While 
SUDAAN has been linked to SAS, the future development 
of WesVarPC, which was originally developed by the 
research corporation Westat, has been turned over to SPSS. 
Further, the survey routines in STATA are part of a larger 
statistical analysis package. As with mergers in the general 
business world, along with product and service integration, 
the future trend for survey data analysis packages is to 
become part of an omnibus statistical package. The deve- 
lopment and maintenance of statistical packages, for survey 
research or for a wider context, is a time-consuming 
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enterprise requiring a substantial capital investment. This 
can only be done by a well-financed organization. 

SUDAAN, STATA and WesVarPC, along with the 
software packages GES from Statistics Canada and another 
named CLAN, have been recently reviewed and evaluated 
in Bergdahl, Black, Bowater, Chambers, Davies, Draper, 
Elvers, Full, Holmes, Lundqvist, Lundstrom, Nordberg, 
Perry, Pont, Prestwood, Richardson, Skinner, Smith, 
Underwood and Williams (1999), SUDAAN and STATA 
have also been evaluated by Cohen (1997). Among three of 
the packages reviewed (STATA, SUDAAN and 
WesVarPC), SUDAAN appears to have the most options. 
For example, Bergdahl et al. (1999) note that SUDAAN 
carries out variance estimation for complex statistics using 
any one of Taylor linearization, jackknife and balanced 
repeated replication. WesVarPC covers jackknife and 
balanced repeated replication, while STATA relies solely 
on Taylor linearization. So far, none of the packages does 
variance estimation using the bootstrap. It may just be a 
matter of time before this technology is incorporated into 
these packages. For some of its public use sample files, 
Statistics Canada provides bootstrap variance estimation 
procedures in SAS code. These procedures, however, are 
specific to the surveys in question. 


7. MODELS IN SAMPLING 


Models have come in and out of favour among sampling 
practitioners. Due to Neyman’s (1934) pioneering work, the 
paradigm of randomization and the randomization distribu- 
tion was paramount until the 1960’s. However, the use of 
models did not disappear during the intervening years. 
Cochran (1946), for example, used models to study certain 
sampling designs and was able to conclude that systematic 
sampling was a good design to use under certain population 
structures. The 1960’s debate over models arose out of the 
questioning of the foundations of sampling initiated by 
Godambe (1955). Since then the use of models has not only 
crept back in to sampling theory but has flourished 
substantially. 

Since the 1960’s the use of models in sampling has gone 
in several directions. At the same time, the practical and 
general use of models in survey estimation and analysis is 
only feasible with high speed computing and the appro- 
priate software. In keeping with the theme I have been 
following here, I will take a very narrow approach to 
models by tying their usage to computing technology. 

Several model-related methodologies have been comput- 
erized, either through the provision of numerical examples 
to illustrate the use of the methodology or through simula- 
tion studies to examine how the methodology works. At the 
present time there is only one model-related approach that 
has matured to the point where a general package program 
is available. This is the model-assisted approach that 
C.-E. Sirndal has taken over several years resulting in 
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generalized regression estimation or GREG. The bulk of the 
work is summarized in Sarndal, Swensson and Wretman 
(1992). The work was initially motivated by the debates 
over the foundations of sampling. Under a model, a best, in 
some sense, estimator of a finite population parameter can 
be derived. Those on the side promoting randomization 
inference pointed out that when the model fails the 
associated estimator can perform very poorly. The solution 
propounded by Sarndal was to obtain the estimate under the 
model and then to adapt it in such a way that it would 
remain consistent and perform adequately under the ran- 
domization distribution. It is an attempt to obtain the best of 
both worlds. Generalized regression estimation, as well as 
several other estimators, have been programmed into GES, 
a generalized estimation system developed at Statistics 
Canada. This SAS-based software is aimed at the descrip- 
tive side of surveys rather than the analytic and is described 
in Estevao, Hidiroglou and Sarndal (1995). It is a package 
that could easily catch on under the right conditions. 


8. CONCLUSIONS 


Developments in sampling research are inextricably tied 
to computing and computational methods. Where research 
is headed will be guided, in part, by computer develop- 
ments. What the immediate future holds for computing is 
greater speed and greater storage capacity so that packages 
can become bigger and more comprehensive. Generally 
acceptable practices in survey estimation and the analysis of 
survey data will be determined by the contents of generally 
available computer packages for survey sampling. On the 
research methodology side, new methodology will continue 
to be increasingly computer intensive. One other foresee- 
able development is the explosion of the internet. As a 
result of this explosion, several complete survey datasets 
are now easily available via the web. The extensive testing 
of new methodology on a variety of real surveys prior to 
publication of the methodology may soon become the norm. 
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The Past is Prologue 
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ABSTRACT 


Mahalanobis provided an example of how to use statistics to enlighten and inform government policy makers. His 
pioneering work was used by the US Bureau of the Census to learn more about measurement errors in censuses and surveys. 
People have many misconceptions about censuses, among them who is to be counted and where. Errors in the census do 
occur, among them errors in coverage. Over the years, the US Bureau of the Census has developed statistical techniques, 
including sampling in the census, to increase accuracy and reduce response burden. A root-mean-square-error model was 
developed to estimate the joint effects of variance and bias in the census. The model is used in this paper to look at the joint 
effects of response variance, adjustment of the bias caused by the undercount, and the use of sampling for follow-up. 


KEY WORDS: Censuses; Mahalanobis; Root-mean-square-error model; Sampling in the census. 


1. INTRODUCTION 


Perhaps it has always been so — that statistics, as a body 
of information, does not always support the actions that 
politicians want to take. In some countries, data from 
censuses are not made public, because knowledge is power. 
However, in our society, the power of statistics is used to 
inform us about needs for action, or how well we’re doing 
as a country, or as the basis of comparison among groups. 
We are used to seeing and trusting statistics on an everyday 
basis, though most of us give little attention to how they are 
produced, by whom, and at what cost. 

Over the last few decades, there have been many issues 
where statistics and politics have been in conflict. Employ- 
ment and unemployment data are often used by politicians, 
especially in an election year. If the unemployment figures 
are low, the incumbents cite that figure and take the credit. 
If the employment figures show that many new jobs are 
being created, that number is cited. Either political party 
can use these data to make whatever political points seem 
salient. An attempt by the Nixon Administration to restrict 
access to these data led to new protections, such that the 
employment and unemployment data are released on the 
first Friday of every month by the Commissioner of the 
Bureau of Labor Statistics at a meeting of the Joint 
Economic Committee on Capitol Hill. 

The definition of poverty is currently under discussion. 
When the poverty measure was invented by Molly 
Orshansky, there were not the large transfer payment 
systems that exist today. Because of income received or 
benefits paid, poverty today does not mean what poverty 
did 30 years ago. However, each political administration 
watches the poverty numbers very closely. These numbers 
were used by critics of the Reagan Administration to illu- 
strate the growing burden of the poor in an administration 
that was alleged to be more interested in serving the rich. 
That Administration argued that by including medical 
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benefits and other transfer payments, the poor were better 
off than before. 

Probability samples of the U.S. population are now used 
to study sexual behavior. Much of our information on 
sexual behavior goes back to Kinsey. The National Opinion 
Research Center (NORC) at the University of Chicago has 
conducted two large surveys of sexual behavior in the U.S. 
One of these, Sex in America, (Michael, Gagnon, Laumann 
and Kolata 1994) reported on a national sample of persons 
aged 18-59, and was not funded by the government. The 
second researched the sexual behavior of adolescents and, 
in both cases, federal funding for these studies was 
questioned because powerful constituencies did not want 
the subject matter to be examined. The second study was 
finally funded by the government. 

Privacy issues abound. For example, there is broad con- 
cern about the confidentiality of individual medical records 
and the need for researchers to access them. Privacy issues 
for groups are less widely recognized. Certain groups may 
not want to report fully in a decennial census or survey 
because they do not want to attract attention. Though 
people who are in the country illegally are supposed to be 
included in the census, many of them fear that government 
authorities looking at block statistics could use the informa- 
tion to raid certain blocks. 

My last example here of issues in which politics and 
statistics are having a disagreement, is the decennial census. 
For decades, an undercount in the census and its differential 
impact on minority populations has been well-documented. 
The Census Bureau has studied this issue for years and now 
has the statistical tools and methods to represent the un- 
counted individuals in the census totals. Yet this “adjust- 
ment” is opposed by many politicians because of an antici- 
pated effect on the drawing of election district boundaries. 
However, the uses of the census extend far beyond appor- 
tionment and redistricting. The battle before the 2000 
Census has been unusually intense. 


Barbara A. Bailar, National Opinion Research Center, 1155 East 60" Street, Chicago, Illinois, U.S.A. 
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Given these instances in which politics and statistics are 
confronting each other, it is useful to step back in time to 
review the contributions of Mahalanobis to the government 
of India. His methods were used successfully by the U.S. 
Census Bureau to learn much of what we know about errors 
in the census. I will review Mahalanobis’ contributions, 
then return to a discussion of the census, the statistical tools 
currently used in the census, additional tools that could be 
used, and then conclude with a plea for Congress and the 
Census Bureau to follow the tradition of continuous im- 
provement in the census through the use of statistical tools. 


2. THE MAHALANOBIS LEGACY 


Mahalanobis played an important role in the methodo- 
logy we take for granted today. He was trained to teach 
physics, but became increasingly interested in statistical 
problems and then in building the Indian Statistical 
Institute. His work on the utilization of interpenetrated sub- 
samples of the population was innovative, and gave great 
impetus to research on the effects of interviewers on survey 
and census statistics. He paid great attention to the need for 
pilot studies to test the implementation of survey tech- 
niques. As time went along, he enlarged his interests from 
sampling and surveys, in which he provided much needed 
information to the government, to planning and economic 
development. He was appointed Honorary Statistical 
Advisor to the Cabinet in January, 1949 and placed in 
charge of the Central Statistical Unit in the same year. The 
central role of statistics in government planning was, no 
doubt, due to the force of the man himself as well as his 
research findings. He saw the role of statistics as a system 
to serve the cause of planned development and envisioned 
’ a feedback arrangement between statistics and planning 
(Rudra 1996). 

The particular contributions I wish to stress today are his 
major roles in sample surveys and in measuring error of all 
kinds — errors of observation, errors of measurement, 
sampling errors, copying errors, printing errors. Much of his 
early work on showing the variability in statistics caused by 
interviewers was in crop statistics (Mahalanobis 1950). He 
was one of the first to say, and then show, that the overall 
error in survey statistics was not just sampling variance but 
also the variance arising from the human element. One way 
to study such errors was by the use of interpenetrated 
subsamples. In the words of Mahalanobis, 


“When two (or more) samples are drawn from the 
same population and covered according to the 
same survey design, the results based on the 
different samples are equally valid, even though 
they are derived by different operational units; and 
divergences between the different sets of estimates 
supply directly some idea of the margin of 
uncertainty.” (Mahalanobis and Lahiri 1961) 
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Mahalanobis demonstrated that statistics based on 
samples were at least comparable to, and often more accu- 
rate than statistics based on a census, in the 1940’s, when 
sampling was still not fully accepted. He believed, as many 
of us now do, that samples can be better controlled than can 
a census. He stated (Mahalanobis and Lahiri 1961) that the 
magnitude of discrepancies found in a census of jute pro- 
duction made it appear that a census may not provide 
accurate estimates for small areas. The random component 
of the non-sampling error may add enough error that results 
for a large area may be no different from those obtained by 
a sample survey. What holds for a large area does not 
naturally follow for small areas. 

The U.S. Census Bureau used Mahalanobis’ techniques 
to learn more about the underlying variability of census 
numbers. 


3. WHAT DO PEOPLE THINK A CENSUS IS 


To most people, taking a census means that enumerators 
go out and count everyone. There are three things that 
people seem to think about censuses. One is that everyone 
is counted. A second is that an enumerator sees everyone. 
A third is that the census is without error. Let’s look at 
these one by one. 

Often, everyone is not supposed to be counted in a 
national census, and who should be counted varies from 
country to country, and over time within a country. For 
example, military personnel and their families located 
outside of the country could be counted or not. Civilian 
aliens temporarily in the country as seasonal workers could 
be counted or not. From these illustrations one can see that 
a primary necessity in census-taking is defining the scope 
of the census. 

So, by definition, certain groups of people are not to be 
counted in the census. This is by design of the Census 
Bureau. Other people make individual or family decisions 
not to be counted in the census. In earlier times, some 
families did not report children who suffered from some 
diseases or retardation. Some people who have had unfor- 
tunate episodes with the legal system may decide not to be 
counted. These may be people who are in the country 
illegally, those who are hiding from law enforcement, and 
those who fear, for whatever reason, the consequences of 
being counted. In 1990, there were people who said they 
would not be counted because they thought the census was 
too intrusive. 

Finally, there are people missed, not by design but by 
accident. Perhaps they lived in buildings that were missed, 
perhaps they lived on the street and were missed. Perhaps 
they were away during the census period. During 1998 
there were many reports of how much harder it was to 
survey people who live in gated communities. It may be that 
some of these people are missed because of the overzealous- 
ness of the community guards. In some communities good 


Survey Methodology, June 2000 


maps are unavailable or not updated, so groups of people 
may be missed. 

In any case, not everyone is counted in a census and 
never was. 

The second myth to be refuted is that an enumerator sees 
everyone and knows who should be in the census or not. 
This never happened, even in the early censuses in the U.S., 
when U.S. Marshals took the census and the country was 
much smaller. In fact, early censuses were of households, 
not of individuals. This means that there were no questions 
asked of individuals but instead there was interest in how 
many people were in the household, how many were men 
and how many were women, how many were in different 
age groups, and so forth. The totals were posted in public 
places. Starting in 1880 the canvasser method of taking a 
census, where enumerators went from door to door, came 
into being. It is this kind of census that made some believe 
that an enumerator saw everyone. However, a single house- 
hold member usually responded for the whole family. The 
enumerator did not see those who were sick, those at work, 
those who were away temporarily, or those who were, for 
some reason or another, not in the room when the 
enumerator visited. 

Though the enumerator-type census was an improvement 
over one taken by the marshals, research using interpene- 
trated subsamples showed that census enumerators still 
added a considerable amount of error to the census 
Statistics. The enumerators were influenced by their own 
expectations and by responses of others in their enumera- 
tion district. Also, some did not understand the instructions 
and reported things incorrectly. An experiment in the 1950 
census showed that enumerators added considerable varia- 
bility to the census statistics (Hanson and Marks 1958). 
Indeed, the statistics gathered from a census had the same 
level of variability, due to enumerators, as a 25-percent 
sample. This is the main reason the Census Bureau turned 
to the use of self enumeration in the 1960 census and 
progressively expanded it in later censuses. Now, if a 
household receives the census form by mail, fills it out, and 
sends it in, and no errors require resolution, no enumerator 
will call at the household. 

The third myth is that census taking occurs without error. 
No one who now works on censuses or surveys believes 
that, but other people do. The Census Bureau encourages 
that belief by publishing data down to the last digit. For 
example, the population of the United States in 1990 was 
reported and published as 248,718,301 in the Statistical 
Abstract. 

Even some of those who have worked closely with a 
census cannot see it as a statistical process that carries with 
it a certain amount of error. Because the error is not 
routinely quantified and published along with the census 
numbers, some cannot believe the error exists. Some 
persons working in the Population Division of the U.S. 
Census Bureau in the 1940’s and 50’s believed that the 
census was the best way to learn about any subject, and that 
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sample surveys were inferior. Repeated demonstrations of 
accuracy in survey results and of bias in census data did not 
change their minds. 

Anyone who comes into regular contact with the census 
now knows that there is error in the data. First, though 
sampling variance cannot occur for items collected on a 
100-percent basis, there may still be substantial response 
variance introduced by effects of enumerators, respondents, 
and coders on census data. Second, bias affects responses 
to many census questions even when a person is correctly 
counted. Bias also affects counts when enumerators do not 
count everyone. The Census Bureau conducts an evaluation 
program as part of every census, documents the amount of 
error, and uses those data to attempt to improve the next 
census. 

Large groups of people are affected by census error. The 
undercounting bias affects minority populations and 
children at a much higher rate than other populations 
(Edmonston and Schultze 1993). Thus, communities that 
are largely African-American, Hispanic, or American 
Indian are underrepresented in distributions of potential 
power and money, while those statistics that are based on 
children under 10 are subject to a large error. 

Over the years, the Census Bureau has reported 
numerous studies looking at the balance between cost and 
accuracy. One mentioned before is the use of self- 
enumeration. At smaller levels of population, the effect of 
response variance, primarily caused by interviewers, was 
very high. Just as with sampling error, as the size of the area 
increased, and the number of enumerators who collected the 
data increased, the effect lessened. When the mail return 
rate was close to 80 percent, the response variance 
decreased to about one-quarter of that of a 25-percent 
sample (Bailar 1969). 

Thus, commonly held images of the census are not 
always true. Also, the census is not always the same. The 
Census Bureau has made many changes in census taking 
since the first census in 1790. The number of questions, the 
kinds of questions, who is counted and where, who does the 
counting, how people are assigned to a geographic domain, 
how missing characteristics are handled, and the gradual 
increase of asking most questions of a sample have changed 
over the years. The next section shows how the use of 
statistical tools has changed the census in this century. 


4. DEVELOPMENT OF STATISTICAL 
TOOLS IN A CENSUS 


Two elements have changed the methods of the U.S. 
Decennial Census considerably since 1940: the use of com- 
puters; and the use of statistical techniques. At times, the 
two elements have complemented each other, for example 
in the fast processing for imputation of missing data using 
a “hot deck” procedure. While computers have profoundly 
affected the census, the remainder of this discussion will 
focus on the statistical methodology. 
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One of the major advances starting in 1940 has been the 
use of sampling in the census. In 1940, as documented by 
Waksberg and Hanson (1965), there were three major uses 
of sampling. One was for the collection of data deemed 
supplementary to the main census questions. Questions 
such as mother tongue, veteran status, and fertility were 
asked of a 5-percent sample. A second use was for certain 
analytic studies requiring clerical transcription and coding. 
To avoid a long timespan for the transcription and coding 
to take place, a sample of census questionnaires was 
selected and the transcription and coding occurred only for 
them. A third use was for the verification of large-scale 
clerical operations such as editing, coding, key-punching, 
and so forth. Prior to 1940, all verification was on a 100- 
percent basis. 

To describe the next leap forward, Waksberg and 
Hanson said: 


“A major step forward in the use of sampling in 
census work took place in the 1950 Census of 
Population and Housing. This grew out of a pro- 
found change in attitude regarding the role of 
sampling. Whereas in 1940 sampling had been 
considered applicable only for items of supplemen- 
tary and secondary interest, in 1950 the entire 
range of census activities was examined to deter- 
mine, on a logical basis, where complete counts 
were necessary and where samples could provide 
adequate information.” 


The increased use of sampling for population characte- 
ristics, for sample tabulations, and for verification was 
successful and evaluations showed that, even with the 
_ addition of sampling error, overall error was less than if 

earlier techniques had been used with no sampling. This 
was a reinforcement of the lesson learned earlier by 
Mahalanobis. 
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During the 1950 Census, the Bureau did a great deal of 
research to learn the effect of response biases and response 
variances on census data. Waksberg and Hanson declared 
that it was misleading to assume that the census, without 
sampling, was without error. In 1950, an experiment was 
conducted to estimate the effect of census enumerators on 
census data. By using the method of interpenetrated sub- 
samples introduced by Mahalanobis, pairs of adjacent 
census areas were merged and assignments to the enume- 
rators were randomized. Since the assignments were over 
the same area, differences between enumerators did not 
reflect differences in the type of area. The main finding of 
the study was that a full census in which enumerators went 
door to door to collect the census information had response 
variability that made the census the equivalent of a 25- 
percent sample (Hanson and Marks 1958). Using that 
result, as well as studies of biases 1n various census items, 
Waksberg and Hanson formulated a model in which census 
results were subject to a relative response bias of 6 percent 
and a response variance equal to the sampling variance of 
a 25-percent household sample. They used this model to 
generate Table 1 which shows the magnitude of total error 
in census data with and without sampling. 


The authors point out that for a characteristic describing 
500 individuals in an area of 2,500 people, the increase in 
the total root mean square error arising from sampling 
variability is only about 25%. For larger areas and larger 
cells, the additional error due to sampling is even smaller. 

These data were studied carefully before the decision to 
increase the use of sampling in the 1960 Census. In 
practice, sampling made even greater gains than those 
anticipated by the model. The authors state “Thus for a 
great many published statistics, the reliability was better 
with the use of sampling than would have been possible 
otherwise.” (Waksberg and Hanson 1965.) 


Table 1 
Expected Root Mean Square Error (RMSE) of Estimated Cell Frequencies for Individual Items Based on a Complete Census 
and On a 25-Percent Sample of Households 


Area of 2,500 Population having 


Area of 10,000 Population having 


Area of 50,000 Population having 


RMSE based on RMSE based on RMSE based on 
Cell Complete 25-percent Cell Complete 25-percent Cell Complete 25-percent 
Frequency Census Sample Frequency Census Sample Frequency Census Sample 

12 7 10 50 1 20 250 34 46 

50 14 19 200 30 40 1,000 85 105 

125 22 31 500 52 67 2,500 180 200 

500 49 62 2,000 140 160 10,000 620 650 

1,250 89 102 5,000 320 330 25,000 1,520 1,530 
Note 1: Sea a assume a relative response bias of 6 percent and response variance equal to the sampling variance for a 25-percent 

sample. 

Note 2: eae of the results (cell frequencies) is measured by a certain kind of average of the actual errors that would occur, the root 


mean square error (RMSE). A useful working rule would be to assume that approximately two-thirds of all results of a census or a 
sample would differ from their true cell frequencies by no more than their RMSE’s. 
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A large-scale evaluation and research program of the 
Decennial Census program began in the 1950’s and is now 
an integral part of the Census. Part of the program tests new 
methods for possible use in the following census and part of 
it focuses on the evaluation of the current census. It was as 
part of this program that the Bureau started measuring the 
undercount in the census. It was also this program in which 
the response variance due to enumerators was measured 
before and after the advent of self-enumeration. (In 1960, 
after self-enumeration was introduced, the response 
variance decreased to 1/4 of the 1950 level.) Since mail- 
back rates have decreased substantially since 1980, that 
variance may have increased again, perhaps substantially. 

Other studies included research on alternative ways to 
measure the undercount, record checks to measure the 
accuracy of census data, and a study of using the Post Office 
not only to deliver census questionnaires but to notify the 
Census about missed addresses and duplicate forms. 

Sampling is now used extensively to control the quality 
of the large-scale clerical tasks associated with the census. 
In past censuses, verification was usually dependent, in 
which the verifier reviewed the coder’s work and deter- 
mined whether the correct codes had been assigned. The 
Bureau planted errors and found that dependent verification 
missed as many as half of the errors. This and other 
research caused the Bureau to develop independent verifi- 
cation, in which records are assigned to three coders who 
do not see each other’s work. A “majority rule” is used to 
determine the best code, and statistics about such errors are 
used to improve the process and to identify substandard 
performance. 

Imputation was also a necessary tool developed for use 
in the census. To keep within time and budget parameters, 
the Bureau developed a “hot-deck” imputation system, 
based on the assumption that people who live in proximity 
are likely to resemble each other for many characteristics 
such as educational attainment and income. Another kind of 
imputation was also used in 1970, 1980, and 1990 to deal 
with a small, residual set of addresses left on the mailing list 
with no information about whether or not they were 
occupied. No one answered the door, nor did neighbors 
know if anyone lived there. Thus, based on a model that 
assumed a high correlation between the characteristics of 
neighboring households, the Bureau imputed occupancy or 
vacancy status, and to those imputed as occupied, a number 
of people were imputed. In 1980, only 762,000 persons 
were imputed, about .003 of the total census count, but they 
were not spread evenly over all the States. As a result of the 
imputation, Indiana lost a Congressional seat to Florida. 
However, it should be acknowledged that doing nothing 
about the unclassified units would have been equivalent to 
imputing them all as vacant. There was information avail- 
able that showed that over half of these units could be 
expected to be occupied so the data based on imputation 
were more accurate than data based on counts alone with no 
imputation. 
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5. ADDITIONAL USES OF STATISTICAL 
TOOLS 


Statistical tools can be used to correct the census for the 
undercount. The Waksberg-Hanson root mean-square error 
model estimates the amount of error in the census assuming 
a relative response bias in the overall census of 2 percent. 
(The 1990 estimate was 1.6 percent.) Also assume a 
response variance in both the adjusted and unadjusted 
census equal to one-fourth the sampling variance of a 25- 
percent sample. That estimate may now be too low since 
decreasing mail-back rates have driven enumerator vari- 
ances higher. However, to be on the conservative side, we 
shall use the 1960 and 1970 measurements. 

The model is the simple mean-square error model used 
frequently by the Census Bureau. 


MSE(f) = Var(T) +B? 


Assume T is a cell size or a size of interest in the census in 
an area where N is the population size. T= NP where P is 
the proportion of the population having a certain 
characteristic. B is the bias in the census count. So, for 
example, in an area of 2,500 people, one might be 
interested in knowing the number of children under 10 
years of age. N = 2,500 and T= NP. 
Now the variance of an estimated proportion, p is: 


N-n 1 
Vi = dein P 
(Vice 


If we have a 25 percent sample, this reduces to 


eal 3PO 
V Eee 2) @ ere tau 
(p) Ae Q < 


V(T) = V\Np)=N?V(p) = 3NPO 
= 370 


Relative bias = (.02) so Bias = .02T 

Now we are dealing with a census, so there is no 
sampling variance, but the response variance is equal to 1/4 
of what the sampling variance would be. So 


MSE(T) = (.027) + (.25)(3)TO 


and 
RMSE(T) = y(.027)* + (.25)(3)TO 


This formula has been used as the basis of the calculations 
in Table 2. For an unadjusted census, RMSE(7) would have 
both the bias and variance components. For an adjusted 
census, the relative bias is zero, so only the response variance 
term remains. However, this analysis presumes that the 
adjustment factors themselves are free from any kind of 
variance and bias, and that the same adjustment factors can 
be uniformly applied within the demographic groups. 
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For example, Table 2 shows that for a total of 500 in an 
area of 2,500, the RMSE for an unadjusted census is 20 
while the RMSE for an adjusted census is 17. For the unad- 
justed census, the contribution from the bias term is small, 
[(.02)(500)]*=100. The contribution from the response 
variance is (.25)(3)(500)(.8)=300. So RMSE =y400=20. 
For the adjusted census, the bias term, 100. is removed, so 
the RMSE=/300=17. However, if one considers that the 
estimated bias term has both variance and bias, there may 
be little difference between the adjusted and unadjusted 
results for a small area. As the total, 7, gets larger, the bias 
term is more dominant, and the adjustment removes more 
error. 

Table 2 shows that for a small area of 2,500 persons 
there is no gain for small totals, but a gain of 43 percent in 
accuracy for a large total of 1,500 persons. In a somewhat 
larger area of 10,000 persons, there is little reduction in 
error until a total of 1,000 is of interest, where there is a 
gain in accuracy of 21 percent and for a large total of 5,000, 
there is a gain of 61 percent. Thus, if we were talking about 
the number of men or women in an area of 10,000, a total 
that might be expected to be around half the population, 
there would be a large gain in the accuracy of the total from 
using adjusted census figures. For an area of 50,000 the bias 
term dominates the mean square error, even at smaller totals 
such as 1,000. Here the gain is 21 percent, which grows to 
81 percent for a very large total of 25,000. 

This illustration shows is that adjusting the census does 
not add to the error of the census, even for small areas and 
small cells, if one assumes that the bias term is measured 
without error. For smaller area sizes and smaller cells, the 
response variance dominates the mean square error, but the 
total error is never less than the response variance. When 
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the census is adjusted, the bias term goes to zero, and the 
gains in accuracy are dramatic. 

One virtue of this model is that it was developed by the 
Census Bureau long before the current debate on adjust- 
ment grew heated. It was used to disabuse people of the 
idea that the census cells have no error. It was used success- 
fully to show critics that having most of the census 
questions answered by only a sample would not hurt the 
data unduly. Such a tried and true census model now shows 
the real value of adjustment. 

Table 2 used the relative response bias of 2 percent based 
on the 1990 Census overall estimate of the undercount of 1.6 
percent. However, since the undercount hits minority 
populations harder, let’s look at a comparison of an adjusted 
and unadjusted census in which the relative bias is 4 percent. 
(The 1990 estimates of the undercount were 4.4 percent for 
African-Americans, 4.5 percent for American Indians, 5.0 
percent for Hispanics, and 2.3 percent for Asians.) 

Table 3 shows the RMSE for minority communities for 
the sizes 2,500, 10,000 and 50,000. Though the RMSE’s for 
the adjusted census stay the same, since the bias has been 
removed, the unadjusted RMSE’s are considerably larger. 
The gains in accuracy from an adjustment are much larger in 
minority communities, as one would expect. For example, as 
shown above, the error in the number of males in a non- 
minority community of 10,000 would be about 109 unad- 
justed and 43 adjusted. In a minority community, the errors 
are 205 and 43 respectively. In a larger area of 50,000 the 
improvement is dramatic even for a small cell of 1,000. 

Now, suppose we repeal the 1976 law that specifies that 
there shall be no sampling for the apportionment numbers. 
Think about a census in which, after a certain date, the 
housing units not returning census forms are sampled. 


Table 2 
Expected Root Mean Square Error (RMSE) of Estimated Cell Frequencies for Population Estimates Based on a Census 
with No Adjustment for Undercount and with Adjustment 


Area of 2,500 Population having 


Area of 10,000 Population having 


Area of 50,000 Population having 


RMSE based on RMSE based on RMSE based on 
Cell Unadjusted Adjusted Cell Unadjusted Adjusted Cell Unadjusted Adjusted 
Frequency Census Census Frequency Census Census Frequency Census Census 

15 3 3 50 6 6 250 LS 14 
50 6 6 100 9 9 500 22 19 
100 9 8 200 13 12 1,000 34 27 
500 20 17 500 21 19 2,500 65 42 
750 25 20 1,000 33 26 5,000 116 58 
1,000 we ZI 2,000 33 35 10,000 214 77 
1,500 37 21 5,000 109 43 25,000 509 OT 
Note: | Computations assume a relative response bias of 2 percent in the unadjusted census and 0 percent in the 


adjusted census. There is a response variance in both the adjusted and unadjusted census equal to 1/4 the 


sampling variance of a 25 percent sample. 
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Table 3 
Expected Root Mean Square Error (RMSE) of Estimated Cell Frequencies for Population Estimates in African-American, 
American Indian, and Hispanic Communities Based on a Census with No Adjustment for Undercount and with Adjustment 


Area of 2,500 Population having 


Area of 10,000 Population having 


Area of 50,000 Population having 


RMSE based on RMSE based on RMSE based on 
Cell Unadjusted Adjusted Cell Unadjusted Adjusted Cell Unadjusted Adjusted 
Frequency Census Census Frequency Census Census Frequency Census Census 
15 3 3 50 6 6 250 17 14 
50 6 6 100 9 9 500 28 19 
100 Q 8 200 15 12 1,000 48 Zul 
500 26 17 500 21 19 2,500 109 42 
750 31 20 1,000 48 26 5,000 208 58 
1,000 45 21 2,000 87 35 10,000 407 77 
1,500 64 21 5,000 205 43 25,000 1,004 97 
Note: Computations assume a relative response bias of 4 percent in the adjusted census and 0 percent in the 


adjusted census. The response variance in both the adjusted and unadjusted census equal to 1/4 the sampling 


variance of a 25 percent sample. 


In this model, there are two components of variance, the 
response variance and the sampling variance. The sampling 
variance is based only on the nonresponse universe. 

Let R be the nonresponse rate, and M the population of 
nonresponse households. Then = RN. The total for which 
we are trying to estimate the sampling variance is S = PM. 
The relationship between S, the sampled part of the total, 
and 7, the total, is through R. S= PM= P(RN) = RT. 

So the sampling variance = 3MPQ = 3PQORN, assuming 
a 25-percent sample of the nonrespondents. This sampling 
rate could easily be changed for a larger rate, but for 
purposes of illustration, it suffices. 

In Table 4, there are three contributors to the RMSE. 
Two of them are the terms we saw in the earlier description 
when sampling of the non-mail returns was not a conside- 
ration. Now we have a third term, expressing the sampling 
variance arising from the sample of non-mail returns. In an 
adjustment, only the bias term goes to zero, while the two 
variance terms remain. Each of the variance terms gets 
smaller as the cell size gets larger, but they do not vanish. 

Table 4 shows the RMSE’s for a census with no 
sampling of non-mail return households, with and without 
adjustment, for a 25 percent sample of non-mail return 
households when only half of the population mails them 
back and when 70 percent mail them back for the three 
sizes of area we have looked at before: 2,500 population, 
10,000 population, and 50,000. The no sampling case is 
what we will have in the 2000 Census because the use of 
sampling for follow-up is prohibited. Look first at Section 
A for a population of 2,500. Where there is no sampling of 
non-mail return households, we see the numbers from Table 
2. When half of the population mails back the census form, 
and the remaining half is sampled, the variance component 
keeps the adjusted and unadjusted RMSE’s very close 
together. At maximum, there is a 20 percent reduction in 


error. There is somewhat more gain when the mailback rate 
is .70 and only 30 percent of the remaining population is 
sampled. The maximum gain in this case is 28 percent. 

Small areas, such as those of 2,500 may be greatly 
affected by sampling, especially at a 25-percent rate if the 
mailback rate is low. Whether a decrease in accuracy is 
acceptable depends on the uses for the data. Since provi- 
ding small area data is an important objective of the census, 
it may be that there would need to be a much larger 
sampling rate, if not complete follow-up for small areas. 
The Census Bureau has done this before with some 
characteristics, such as income, so that there would be less 
variability in the income data for areas of 2,500 or fewer 
persons. Following that same principle, it could be specified 
that there would be no sample follow-up in places of 2,500 
persons or fewer, and variable follow-up rates depending on 
place size. Another strategy would be to use the information 
abundantly available about coverage error and to specify 
larger samples in places that have characteristics highly 
correlated with the undercount. 

For areas of 10,000 population, we see a definite impro- 
vement from the adjustment for the bias, but the adjusted 
numbers with sampling are still considerably larger than the 
adjusted figures without sampling. However, if there is no 
adjustment, the sampling adds to the RMSE, but the 
unadjusted numbers are not much different. There is a 15 
percent increase in the RMSE when only half the popula- 
tion returns the census form and an increase of 9 percent 
when 70 percent return it. 

Finally, when we look at an area of 50,000 we see that 
the bias dominates the RMSE for all but the smallest cell 
sizes. When the total we are trying to estimate is 5,000 or 
larger, sampling adds to the RMSE, but an adjustment, with 
sampling, is still superior to unadjusted numbers with no 


sampling. 
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Table 5 is similar, but geared to a predominantly 
minority population. As in Table 3, the relative bias is 4 
percent, reflecting an average undercount rate for mino- 
rities. In this table, the RMSE’s for unadjusted totals are 
much more similar, even for smaller areas, because of the 
larger effect of the bias term on the RMSE. 

The results for areas of 50,000, which exist in most large 
cities, show the devastating effects of not adjusting for the 
large minority undercount. The sampling variance for the 
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larger totals has practically no effect on the RMSE, but the 
improvement from adjustment for all cases, sampling or no 
sampling is 83 percent or higher. The added error because 
of sampling is negligible. 

Unfortunately, in many minority communities, low mail- 
return rates and undercounting occur together. Such 
communities have a 50 percent mail return rate or lower. It 
may be that the sample size will need to be increased in 
these areas. 


Table 4 


Expected Root Mean Square Error (RMSE) of Estimated Cell Frequencies for Population Estimates Based on an Unadjusted Census, 
an Adjusted Census, and on a 25 Percent Sample of Non-Mail Return Households 


A. Area of 2,500 population having RMSE based on 


No sampling of non-mail return HH’s 


25% sample and .50 mailback rate 


25% sample, and .70 mailback rate 


Cell Frequency Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted 
es wes 3 3 6 6 5 5 
50 6 6 11 jLi 9 9 
100 oD 8 15 iS 13 13 
500 20 17 32 30 28 26 
750 25 20 38 34 33 29 
1,000 29 21 42 3] 37 31 
1,500 337) 2) 47 37 43 3] 


B. Area of 10,000 population having RMSE based on 


No sampling of non-mail return HH’s 


25% sample and .50 mailback rate 


25% sample, and .70 mailback rate 


Cell Frequency Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted 

50 6 6 1] 11 9 5 

100 9 9 15 Is 13 13 

200 13 12 aA P| 18 18 

500 21 19 34 33 30 28 

1,000 35 26 49 45 43 39 

2,000 53 35 1p) 60 65 51 

5,000 109 43 125 75 119 64 


C. Area of 50,000 population having RMSE based on 


No sampling of non-mail return HH’s 


25% sample and .50 mailback rate 


25% sample, and .70 mailback rate 


Cell Frequency Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted 

250 5) 14 24 24 21 20 

500 22 19 35 33 30 29 

1,000 34 Di] 51 47 45 40 
2,500 65 42 89 73 80 63 
5,000 116 58 142 101 132 86 
10,000 214 Wa) 241 134 231 SNS) 
25,000 509 97 Veil 168 520 144 
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Table 5 
Expected Root Mean Square Error (RMSE) of Estimated Cell Frequencies for Population Estimates in African-American 
American Indian, and Hispanic Communities Based on an Unadjusted Census, an Adjusted Census, 
and on a 25 Percent Sample of Non-Mail Return Households 


A. Area of 2,500 population having RMSE based on 


No sampling of non-mail return HH’s 


25% sample, and .50 mailback rate 


25% sample, and .70 mailback rate 


Cell Frequency Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted 

15 3 3 6 6 2) 5 

50 6 6 11 itil 9 9 

100 9 8 15 15 13 13 

500 26 17 36 30 33 26 

750 31 20 46 34 42 29 

1,000 45 21 54 37) Sy 31 

1,500 64 21 70 37) 68 31 


B. Area of 10,000 population having RMSE based on 


No sampling of non-mail return HH’s 


25% sample, and .50 mailback rate 


25% sample, and .70 mailback rate 


Cell Frequency Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted 

50 6 6 11 tal ) 9 

100 9 9 IS 15 13 13 

200 15 Wy 22 21 18 18 

500 21 19 38 33 34 28 

1,000 48 26 60 45 56 39 
2,000 87 35) 100 60 95 51 
5,000 205 43 214 75 210 64 


C. Area of 50,000 population having RMSE based on 


No sampling of non-mail return HH’s 


25% sample, and .50 mailback rate 


25% sample, and .70 mailback rate 


Cell Frequency Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted 

250 17 14 26 23 23 21 

500 28 19 39 33 35 29 

1,000 48 Dy 62 47 57 40 
2,500 109 42 124 73 118 63 
5,000 208 58 224 101 218 86 
10,000 407 77 422 134 416 115 
25,000 1,004 97 1,014 168 1,010 144 


6. CONCLUSION 


It has been a tradition for the Census Bureau in the latter 
half of this century to use statistical techniques, where pos- 
sible, to make the Decennial Census more accurate and less 
costly. Using the techniques historically used by the Census 
Bureau, namely a mean-square error model, one can see 
that adjustment does improve census totals, even for small 
areas, when one assumes even a minimal level of response 
variance. One can also see the need for precaution if samp- 
ling is to be used for follow-up. It may be that there should 
be no sampling in places of 2,500 or fewer people, just as 
there is no sampling for certain population characteristics in 
these small places. 


In looking at the current census controversy, it is good to 
remember the spirit of Mahalanobis. Not only did his 
ingenious use of interpenetrated subsamples give us the 
ability to estimate the response variance in census statistics, 
but his insistence that sampling and statistics should be used 
to solve practical problems has been the hallmark of the 
U.S. Census. Some of the most fundamental practical 
problems are those faced by the government and 
Mahalanobis allocated statistical resources for the solving 
of these problems. Likewise, the U.S. Census Bureau has a 
long and rich history of offering practical, cost-efficient 
solutions to thorny census problems. 
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Estimation of Census Adjustment Factors 


C.T. ISAKI, J.H. TSAY and W.A. FULLER! 


ABSTRACT 


A components-of-variance approach and an estimated covariance error structure were used in constructing predictors of 
adjustment factors for the 1990 Decennial Census. The variability of the estimated covariance matrix is the suspected cause 
of certain anomalies that appeared in the regression estimation and in the estimated adjustment factors. We investigate 
alternative prediction methods and propose a procedure that is less influenced by variability in the estimated covariance 
matrix. The proposed methodology is applied to a data set composed of 336 adjustment factors from the 1990 Post 


Enumeration Survey. 


KEY WORDS: Components-of-variance; Small area estimation; Undercount; Decennial Census; Smoothing. 


1. INTRODUCTION 


While the objective of a population census is to record 
data for all individuals, it has long been recognized that this 
goal is not achieved in practice. Post enumeration studies 
associated with the U.S. Census of 1970 and 1980 sug- 
gested that the coverage rate was different for different de- 
mographic groups. See U.S. Bureau of the Census (1988). 

In 1990, a post enumeration survey (PES), using dual 
system (or capture-recapture) estimation, was used to pro- 
duce estimates for 1392 subdivisions of the total population 
of the United States at the time of the 1990 Census. The 
PES sample contained approximately 377,000 persons in 
about 5200 sample blocks. Sample persons were divided 
into post-strata defined by geographic divisions of the coun- 
try, tenure, size-of-place, race, sex, and age, where the two 
tenure classes are owners and renters of homes, and size-of- 
place is a measure of urbanization. The subdivisions were 
called poststrata. The ratio of the PES estimate to the 
Census total, called the adjustment factor, was produced for 
each poststratum. An adjustment factor greater than one is 
associated with an estimated undercount and a factor less 
than one is associated with an estimated overcount. 

Because relatively large sampling variances were antici- 
pated for individual ratios, a smoothing technique based on 
components-of-variance and a regression model was used 
to create the final estimated adjustment factors. The ele- 
ments of the error covariance matrix used in the prediction 
model were estimated with a jackknife algorithm, see Fay 
(1990). 

The explanatory variables in the regression model were 
chosen using a best subsets selection algorithm. Some ex- 
planatory variables were forced into the model. For exam- 
ple, in the Midwest region, the ten explanatory variables 
forced into the model were Black, Hispanic, renter, age 
group 0-9, age group 10-19, age group 20-29, age group 
30-44, age group 45-64, male 10-19 and male 20-64. Most 


variables were indicator variables, but some were propor- 
tions. For example, a variable “percent Black” was used 
when Black and Hispanic were grouped into a single post- 
stratum. Nine other variables were selected for inclusion in 
the model based on a best subsets regression algorithm. The 
variables included mail return rate, substitution rate, type- 
of-place and six race-by-age and race-by-tenure interaction 
variables. The mail return rate is the fraction of Census 
questionnaires returned from the mail distribution, the sub- 
stitution rate is the fraction of Census households that were 
entirely replaced with responding households. 

The smoothing technique was applied to poststrata ratios 
by regions of the country. The adjustment factors were 
designed to be applied to Census counts in the appropriate 
poststrata to create population estimates adjusted for under- 
count or overcount. Hogan (1992) contains an overview of 
the PES. Isaki, Huang and Tsay (1991) provide a detailed 
description of the results of the smoothing of the 
poststratum ratios. 

Fay (1992) in a manuscript discussing the adjustment 
factors constructed from the 1990 PES, identified some dis- 
turbing results. He noted that some of the estimated regres- 
sion coefficients in the model differed considerably depen- 
ding on the form of the estimated covariance matrix used to 
construct the estimated generalized least squares estimator. 
Fay conjectured that large differences in coefficients could 
arise because of an unstable estimator of the error covari- 
ance matrix. Although the estimated error variances were 
smoothed, it was felt that estimated variances of linear com- 
binations might still have large variances. He felt that the 
estimated variances had large variances because the direct 
estimates for many blocks were zero. 

The Secretary of Commerce ultimately decided to use 
the unadjusted counts in the Decennial Census. The possi- 
ble use of adjusted counts for other purposes, such as the 
Bureau’s postcensal estimation program, was left for 
additional study. 
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We explore alternative smoothed estimators for the 
adjustment factors, focusing on the effect of estimating the 
covariance matrix of the vector of the estimated adjustment 
factors. In the empirical part of our study, we construct 
estimates based on the 1990 Census data. 


2. SMOOTHING MODEL 


The model chosen for the construction of predictors is 
the multivariate components-of-variance model. Closely 
related models that lead to smoothed estimators for a set of 
unknowns, have been studied by a number of authors. Fay 
and Herriot (1979) suggested the use of the model in a 
small area estimation procedure. Battese, Harter and Fuller 
(1988) applied the components-of-variance model to crop 
area estimation. Ericksen and Kadane (1985), Cressie 
(1992), and Ericksen, Kadane and Tukey (1989) suggested 
smoothing procedures for census adjustment. Singh, 
Gambino and Mantel (1994) discuss a range of small area 
procedures. Efron and Morris (1972) and Morris (1983) 
contain good discussions of some of the basic theory. 
Kackar and Harville (1984), Peixoto and Harville (1986), 
Fay (1987), Fuller and Harter (1987), Hulting and Harville 
(1991), Ghosh (1992), and Prasad and Rao (1990) discuss 
estimation and variance estimation for such procedures. 
Ghosh and Rao (1994) is a review article. 

Under the multivariate components-of-variance model, 
the vector of true values to be predicted is 


y = XB + w, (1) 


where y is an n-dimensional column vector, X is ann xk 
matrix of observable characteristics, w is an n-dimensional 
column vector of random effects and B is a k-dimensional 
unknown column vector. The vector Y is observed, where 


Y=y +e, (2) 


Y is an n-dimensional column vector and e is the n-dimen- 
sional column vector of estimation errors. In our application 
Y is the vector of estimated adjustment factors. It is 
assumed that 


(w',e’)' ~ (0, block diag {Io”, zy, A) (3) 
War, X,, 1s the covariance matrix of the estimation errors, 


and o” is ; the unknown variance of the random effects. 
A class of predictors of y is defined by 


y = XB +G’(Y - XB), (4) 


where B is a k-dimensional vector and G is an n x n matrix. 
Under model (1) with 


(w’,e’)’ ~ N(0, block diag {Io?, ¥,,}), (5) 
the conditional expected value of y given Y is 
E{y|¥}=XB + Gi(y - Xp), (6) 


where G_ = ZX. 07 and)". = =o? + x,, isthe n xn covari- 
ance matrix of z=w+e. Under the “normal Se 
model defined by (1), (2), and (5) and with the parameters 0”, 
x, B known, the minimum mean square error predictor of 
y is given by the right side of equation (6). 

Generally, some of the parameters are unknown. Con- 
sider first the case in which B is unknown. Let B be an 
estimator of B, where 


B = (X'M?!X)!x’M’!y, (7) 
and M is an n Xn matrix. If M is fixed 


j -y =(1-G)X(B - B) - (1- G)w+ Ge 


=(K -I)w+ Ke, 
where K = (I - G’) X (X’M!X)!X’M'!+G’. Thus, if 
M and G are fixed, 
V{y -y} =K-1I)K-1)'o +KZ_K’. (8) 
If model (1), (2), and (3) holds, and if )\,, and o° are 
known, then replacing B with 
B -(x'nrx)'X’'ELY ©) 
and replacing G with 
G_ ==, 0 (10) 


in (4) defines the best linear unbiased predictor of y. See 
Henderson 0) Harville (1976), and Robinson (1991). 
If &, and o” are algo unknown, it is natural to use esti- 
mators of X,, and o” to construct an estimated best te 
unbiased predictor. Very often, an estimator of &,, 
associated with the procedure used to construct the =a 
mator Y. Then o? is estimated from model (1), (2), and (5), 
treating the estimator of &,, as the true &... 
One substitution predictor is 


§ = XB +6E2)(¥ - XB), (11) 
where 
pe (xt x)" x'zly (12) 
is the estimated generalized least squares estimator of B, 
E-16?+5, (13) 


A 


x, is an estimator of x,» and 6’ is an estimator of 07. 
The estimator of o” can be based on likelihood or analysis 
of variance procedures. Retaining only the terms in the 
Taylor expansion of the error in (11) that are errors in the 
basic estimators, we have 


§ -y+e-H’z+H’X(f -B) 
+ (6? -o*) H’E,)z 


AGYE Ey \ Eo, (14) 
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where H’=%,,2), and G'=1-H’=07E.). If it is 
assumed that Z,, is distributed as a multiple of a Wishart 
matrix pene de _ degrees of freedom, if the covariance bet- 
ween o and oe is ignored, if expectations are computed 
as if 6” and z are > independent, and if expectations are com- 
puted as if z and = are independent, an approximation to 
the variance of y - y obtained from (14) is 


V{y-y}+2%,G+ H'XV,,, X'H BE sel, 4445) 
where 


Vga Mibh-(XEEX) 24,1 x! ho 


ZZ ee 


| eR | ap an ‘HY. 


oo’ 


Ty, dot EE,2: [w(x 5 \) 


l= (X'E Xe XE, 


and V_, = V{6"} is the variance of 6”. The term ¥,,G is 
the prediction covariance matrix if all parameters are 
known. The remaining three terms of (15) are ie contribu- 
tions to the variance due to estimating B, o”, and es 
respectively. The second term in V{} is a crude approxi- 
mation for the increase in the variance of B due to using an 
estimator of XZ, in place of X_ in constructing B. 

If the dimension of 22 is large and the degrees of 
freedom, d,, only slightly larger than the dimension, then 
the second part of the variance of B and the term I’,, can 
make important contributions to the variance. This is parti- 
cularly true if 6? is small relative to the diagonal elements 
of X,,. The Monte Carlo study of the next section demons- 
trates that the contribution to variance approximated by 
these terms can be important. 

A predictor that reduces the effect of the estimation error 
in X,, uses only diagonal elements of %,, in the shrinkage 
component. Let 


¥,=X6,+6’D, (Y - XB.,), (16) 
where 


B,= (XD x) 'x'DZY, 


D., - diag(%,, + 167), 

6” is an estimator of 6” and diag (A) is the diagonal matrix 
composed of the diagonal elements of A. Retaining only 
the leading terms in the Taylor expansion of the error in 


(16) gives 


¥4-y = -(w-Gjz)+HjX(B, - B) 
+(6 -0°)HjDz'z - G;(B,, - D,,) D,'z, (1) 
where D_ = diag{Z_}, G, =D, H,=1-G,, and 


Dy = = diag fo ES Ifw and ‘ are Lalit ‘distributed, and 
if 6 and Dd. _are quadratic estimators, then 6” and Dd. _ are 
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uncorrelated with z. The i-th element of w - oD, ay ao 
uncorrelated with the i-th element of z, but is not neces- 
sarily uncorrelated with the vector z. If this possible cor- 
relation is ignored, if it is assumed that »3) is a Wishart 
matrix ae d, degrees of freedom, and if the correlation 
between 6? and %,, is ignored, an approximation to the 
variance of ¥,-y obtained from (17) is 

V{¥,-y} =HjH,o° +G/5G at H)XV,,X'H , 


d “ee 


iy DP 33aa ‘a D yaaa? a 8) 


where G , =D, aon H,=1-G,, 


mol = Tr z Ty = 
=(X'D_X)'X'D EDL X(X’DLX)1, (49) 


[ys Hyp ohcDisH nie (20) 


I genie eG Ge 
and the ij-th element of Q is 


y= 2cei ox} 0, 22h) ieee 

The term in I’, ,, is an estimator of the contribution to the 
variance due to using pe , to estimate the covariance matrix. 
Expression (19) assumes that the contribution of the error 
in D, to the variance of B can be ignored for large d,. The 
difference between (15) and (18) is that the multipliers i in 
(19) and (20) do not depend on the dimension of Z_. 
Therefore, the error in estimating & makes a smaller 
contribution to the variance. On the other hand, the variance 
of w - G jz, the order one term of (17), will be larger than 
the corresponding term of the error in (14), unless Z_, is 
diagonal. The first two terms on the right of (18) are the 
variance of w - GZ. 


3. MONTE CARLO STUDY 


To examine the variability in the predictors associated 
with variability in the estimation of &,, we conducted a 
small Monte Carlo study. The model for the study is 


Y,=ynJ+wre,, Janae cient (21) 


w ~ (0,1o0’), 
e, ~ ind(0, %,,}, 


where J is the k-dimensional column vector of ones, 
J =(1,1,..., 1)’, w is the k-dimensional vector of random 
small area effects, e, is a vector of errors, and w and e. are 
independent. The model is a simplified version of the model 
defined in (1), (2), and (3). The mean is the constant func- 
tion and, hence, we use p in place of B. To create a vector 
of correlated variables, we define, for k = 8, 
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ey ] 3uy, 

ey, 1.5u,, + 0.4u,, 
ey 0.9u,; + 0.9u,, 
ey | _ 0.9u,, + 1.6u,, 
es 1.6u,, ee 0.6us, ‘ 
&5) 1.0u,, + 1.6u,, 
ey 1.0u,, 

ey 2.83ug, 


where u, are independent random variables. The w,, 
i=1,2,...,8, are NI(O, 0.36) random variables, where 
NI(u, o”) denotes normal independent random variables 
with mean p and variance o”. This configuration gives a 
range of error variances and a range of correlations between 
estimates. 

The estimator of o* used in the Monte Carlo study is 


6? = max {(k-1)7 


i (G - Iho)’ (¥- Iq) -tr {r "k, Ao} 0} (22) 


where tr{ A} is the trace of the matrix A, 


A, =1-k' IS’ 
. = af =] : ae; Sa 
%.=(r - 1) XY) y¥,-y)', 23) 
and 
fi, =k'I’y. (24) 


The estimator 6? is a quadratic estimator closely related to 
the analysis of variance estimator. 

Two predictors were compared in the Monte Carlo study. 
Both are of the form 


¥=y -H’(y - pid), (25) 


where 
ae 
y=r 2. Y.. 


They differ in the construction of H and fi. The first 
predictor i is of the form (11) and uses the full estimated hea 
in Hi and in the estimator of uu. The predictor is called the 
general predictor as an abbreviation for estimated general- 
ized least squares predictor. The general predictor is 


9 ony HAG by), (26) 
where 
H; “ee: >a 
aly =| aes! ait aes ea 
plone srl saa, (27) 


PE yl) ioctl (i (28) 


and A, is the estimated generalized least squares estimator 
of u. 
The second predictor is 


§,7¥ -Hj(y -0,5), (29) 
where 
au ee bs Ds 
Me diag®.., De = =diagZ_, and the estimated p is 


A, = (DEI IDLy. 


This predictor might be called the diagonal predictor 
because only the diagonal elements of S are used in the 
construction. 

The entries in Table 1 are for r = 14. Each sample is 
composed of a random selection of w and a random sample 
of 14 e-vectors. Results are given for errors Uy, ~ NI(0, 2) 
and errors that are centered one-degree-of- freedom chi- 
square random variables. Thus, in both cases the errors 
have zero means and variances equal to two. The mean pu 
was set equal to zero. The second column of Table 1 con- 
tains the variance of the sample mean as an estimator of the w,. 
Column three of Table 1 contains the ratio of the Monte 
Carlo variance of an element of ¥,, where ve is defined by 
(28), to the Monte Carlo variance of the corresponding 
element of y for normal errors. The ratios for elements one 
through four and element 7 are greater than one. The last 
two elements of Y , are uncorrelated with other elements. 
Element seven has a small variance and element eight has 
a large variance. There is a large loss for the predictor 
relative to the simple mean for element seven and a large 
gain for element eight. 

The fourth column of Table 1 contains the ratios of the 
variance of the predictor of (29) to the variance of the mean 
for normal errors. In all cases the diagonal predictor is 
superior to the general predictor defined in (28). The differ- 
ence is relatively constant at about 30%. The diagonal pre- 
dictor is not always superior to the simple mean but the loss 
is small for elements one, three, and seven. On the other 
hand, the gains relative to the simple mean are large for 
elements six and eight. The Monte Carlo variances for both 
predictors are larger than the approximations associated 
with equations (15) and (18) except for element 8. 

It is somewhat surprising that the diagonal procedure did 
better relative to the simple mean for chi-square errors than 
for normal errors. With the chi-square error, the estimated 
mean and estimated variance are correlated. Hence, on the 
average, the large positive mean deviations are pulled to- 
ward the mean by a larger amount than the smaller negative 
deviation. The Associate Editor conjectured, and we con- 
cur, that this is one reason for the superior performance of 
the diagonal predictor. On the other hand, the general 
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prediction procedure is poorer relative to the simple mean 
for chi-square errors than for normal errors. As the last 
column of Table 1 demonstrates, the diagonal predictor 
procedure uniformly dominates both the mean and the gen- 
eral prediction procedure for this parametric configuration 
with chi-square errors. 


Table 1 
Monte Carlo Variance Ratios for Alternative 
Small Area Predictors 
(10,000 samples, r = 14) 


Normal Errors Chi-square Errors 


seu ibirs VO Seal Vp alee! Dect) VO, %)) 
is ads Lal LD) vy. Lon 
1 0.2414 1.277 1.025 1.430 0.899 
2 0.3445 [pAsyy 0.875 yal 0.768 
SO 268 ikesieyy| 1.019 1.480 0.954 
4 0.4771 1.003 0.735 1.099 0.686 
5 0.4113 0.926 0.876 1.016 0.699 
OF H70:5121 0.913 0.677 0.975 0.618 
7 0.1449 1.366 1.006 2.261 0.896 
8 1.1214 0.520 0.384 0.725 0.371 


The Monte Carlo variances of fiy, fig, and fi, as esti- 
mators of p are 0.150, 0.273, and 0.146 , respectively. If 
X,, and o* are known, the variances of fiy, i go and fi, are 
0.149, 0.122, and 0.140, respectively. The use of an esti- 
mated covariance matrix for fi, produced an estimator with 
larger variance than that of the simple mean. 

The predictors are unbiased under the model when the 
errors are normally distributed. The predictors are biased 
with chi-square errors because the sample mean is correlated 
with the sample variance. Table 2 contains the Monte Carlo 
bias divided by the Monte Carlo standard error of the mean. 
The bias of the general procedure is 20% to 50% larger than 
that of the diagonal procedure. In both cases, the squared bias 
added to the variance produces a mean square error for the 
procedure that is about 4% to 10% larger than the variance. 

This small study demonstrates that use of an estimated 
covariance matrix with large variability can lead to predic- 
tors that are less efficient than the simple mean. 


Table 2 
Monte Carlo Relative Bias of Alternative 
Small Area Predictors 
(10,000 samples, r = 14, chi-square errors) 


Ave. Og -w,) Ave. {),, — W,)} 
iy aw Wy eatin 

1 -0.28 -0.19 

2 -0.27 -0.18 

3 -0.30 -0.17 

4 -0.27 -0.18 

5 -0.26 -0.21 

6 -0.29 -0.20 

7 -0.24 -0.20 

8 -0.24 -0.21 
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4. APPLICATION TO PES DATA FOR 
POSTCENSAL ESTIMATION 


4.1 Postcensal Estimation 


The U.S. Bureau of the Census provides annual esti- 
mates of the population of small areas based on the decen- 
nial censuses and on other sources of information. To con- 
sider the possible use of adjusted 1990 Census counts in the 
postcensal estimation process, the Bureau examined the 
PES data and defined a new set of 357 poststrata. 

The 357 poststrata are composed of 51 poststratum 
groups, each of which is subdivided into 7 age-sex categor- 
ies. The seven age-sex categories were (1) both sexes 0-17, 
(2) males 18-29, (3) males 30-49, (4) females 18-29, (5) 
females 30-49, (6) males 50+ and (7) females 50+. The 
factors that define the 51 poststratum groups are race/ethni- 
city (Non-Hispanic White, Black, Non-Black Hispanic, 
Asian, American Indian); tenure (owner, renter); type of 
area (urbanized area of population greater than 250,000, 
other urbanized area, non-urbanized area) and region (West, 
South, Midwest, Northeast). Due to sample size limitations, 
American Indians comprised a single poststratum group and 
Asians were dichotomized into two poststratum groups — 
owners and renters. Of the remaining 48 poststratum 
groups, the first 24 groups reflect a full cross classification 
of categories for Non-Hispanic White. The next 12 groups 
are for Black and provide a full cross classification of 
tenure by region for urbanized areas of population greater 
than 250,000 but otherwise do not provide regional detail. 
The same 12 poststratum groups were used for Non-Black 
Hispanics as were used for Blacks. 

A 357 x 357 covariance matrix was obtained with the 
same jackknife algorithm used for the 1392 poststrata of the 
1990 PES. We denote this raw covariance matrix by )’,,. 
Hogan (1993) provides a detailed description of the 357 
poststrata and gives the motivation for their construction. 


4.2 Regression Model 


We eliminated Asian and American Indian data from the 
smoothing process. Hence, minority refers to the combina- 
tion of Black and Non-Black Hispanic. The data set of 
interest contains 336 adjustment factors and their estimated 
raw covariances. The minority by age-sex interaction was 
included in the regression model after examination of the 
1990 data indicated that the net undercount differential bet- 
ween Black and Non-Black varied by sex and age-group. 
The regression model (1) contains 21 explanatory variables. 
They are: 


ee 


2. X,= indicator variable for age-sex categories: 
j=1,2,..., 6in the order; ages 0-17, male 18-29, male 
30-49, etc. mA A 50+ is the class with no variable) 


= intercept 
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= jndicator variable for renter 


indicator variable for Black 


eae an 
i 


= indicator variable for Non-Black Hispanic 


ny EA gee 


X= indicator variable for type of place: 7 = 10, 11 for 
urbanized area 250,000+ and other urban, respectively 


7. X.= indicator variable for region: j = 12, 13, 14 for 
ortheast, South and West, respectively 


8. X. = indicator variable for minority by age-sex inter- 
action: j = 15, ...,20 for minority 0-17, minority male 
(18-29), etc. 


The variables X,,, X,, and X,, were the 1990 census 
proportions of persons in ‘the poststratumn group in the parti- 
cular region for the Black and Non-Black Hispanic post- 
stratum groups that were combined over regions. 

A refinement was made in model (3) for the empirical 
application. On the basis of preliminary analysis, the speci- 
fied error structure of w, the model error, was changed from 
Dax = 07! to 


ww 


Eyy = Ko; + K,o3, (30) 


where K, is an n xn diagonal matrix with ones for minor- 
ity poststrata and zeros elsewhere and K, isan nxn diag- 
onal matrix with ones for nonminority poststrata and zeros 
elsewhere. The estimated variances are 6i = 0.000506 
(0.000140) and a= 0.000112 (0.000030), where the 
numbers in parentheses are standard errors. The standard 
error of the difference is (0.000141). Hence there is 
evidence that the variances are different for the two groups. 

In our discussion of predictors, we considered two pre- 
dictors, the substitution predictor of (11) and the diagonal 
predictor of (16). It is natural to consider a compromise 
predictor of the form 


= XB, + G.(Y - Xf,) 
=¥ = H/(Y = XB,), (31) 


where 0< g <1, 


A, i G, , Loo | OD. : (1 ‘i 9) Zee) 


Loo i Day Bs oD,, a dd 7 @)%,,» 


and 
Se aK Op kaos. 

The predictor (31) with @ =0 is the substitution pre- 
dictor and the predictor (31) with @ = | the diagonal pre- 
dictor. There should be some 9, 0<@< 1, that gives a 
predictor with smaller prediction variance than either of the 
extremes. 

The PES direct estimate of the total number of persons 
is the weighted sum of the adjustment factors, where the 
weights are the census counts in the post strata. The stan- 
dard error of the direct estimator of the total is relatively 
small and the direct estimator is judged to be the preferred 
estimator of the total. Therefore, the model predictors are 
constructed subject to the constraint that the weighted sum 
of the predictors is equal to the direct estimate of the total. 
Thus, the restriction is 


i=] i=] 


where es is PES direct estimator of the total, a, is the 
census count in the i-th post stratum, and jy, is the final pre- 
dictor. In the actual computations the a, were normalized 
to sum to one. Battese, Harter and Fuller (1988) made an 
adjustment in the predictions to create estimators to meet 
the restriction. Ghosh and Rao (1994) discuss such adjust- 
ments. We use a procedure that permits direct estimation of 
the variance of the restricted predictions. 

We imposed the restriction on the initial predictors by a 
procedure that, approximately, constructed the best predic- 
tors of 335 quantities that are estimated to be uncorrelated 
with Y. 7 Let Z_, be the estimated covariance matrix of 


Yea yay and define 
CY ee 
where 
C =BT, 
to: | 
0 1535)’ 
a=(a vp yge)s 
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I, is the &x& identity matrix, and 0 is a column vector 
containing all zeros. The elements of CY are uncorrelated 
with Y,.. 

If we let ¥ be the model predictor of y, then the model 
predictor of Cy is Cy. If we use the model predictor for the 
last 335 elements of Cy and use Y r as the estimator for the 
first element of Cy, the predictor of y is 


yaw Co ACH’ (¥.+ XB), (32) 


where 
0 0’ 
bX . 
( 0 a 


The estimated variance of y is 


V{¥-y} 


A 


=(1-H, )%,,(1-H, )' +H = H, 


Q” ww 
+C'AC/H/XV,,.X'H, +1, +1 ,,|C’AC'', (33) 


where H’ = C'ACH’, and Ve Ale and ri, are defined 
in Appendix B. The sum of the ae two terms on the right 
of (33) is an estimator of the variance treating H, as a fixed 
matrix. The final term on the right of (33) estimates the 
increase in variance due to estimating the variance. 


4.3. Smoothed Factors 


For the vector of 336 observations, we produced 
smoothed factors using the generalized predictor (32) for 
several values of @. Note that @ = 0 corresponds to the sub- 
stitution predictor and @ = 1 corresponds to the diagonal 
predictor. 

The estimated standard errors of the predictors were 
calculated using the crude variance approximation of 
Appendix B. The average of the ratios of the standard error 
of ¥, to J). for some selected values of @ are given in 
Table 3. The ordering of the ratios is approximately the 
same for the 48 stratum groups as for the original 336 post- 
strata. A poststratum group is formed by combining the 
seven age-sex cells within a given race-by-tenure-by-urba- 
nity-by-region classification. On the basis of these calcula- 
tions, a of 0.5 or 0.6 is the preferred estimator, although 
the estimated differences in efficiencies are not large. Any 
member of the g-class is much superior to the original 
Y-estimator. The average estimated variance efficiency is 
about 400% for the o-predictors, relative to the original 
poststratum estimators. 
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Table 3 
Average of Ratio of Standard Error of vs 


and of ¥ to Stardard Error of Vo5 
a 


336 48 
Predictor Poststrata Poststratum groups 
o=0 1.014 1.045 
o=0.5 0.995 1.001 
p= 0.6 1.000 1.000 
g=0.7 1.006 1.001 
~=0.8 1.014 1.005 
O=k0 1.046 1.037 
Original Y D235 2.294 


Table 4 presents the raw PES estimates, Y, and the Woe 
estimates of net undercount for each of 48 poststratum 
groups. The net undercount is the difference between the 
estimated total population in the poststratum and the census 
count divided by the census count. 

We chose @ = 0.6 as the preferred estimator on the basis 
of the crude standard error ratios of Table 3. The predic- 
tions and standard errors are very similar for @ = 0.5, 0.6 
and 0.7. A 9 greater than zero has advantages over a 9 of 
zero. The accuracy of the numerical calculations should be 
better with @ greater than zero because LZ has larger 
eigenvalues with @ >0 than with = 0. One could make 
a case for using @ = 1.0 because of the simplicity of the 
calculations and of the good estimated relative efficiency. 

The estimated standard errors of the predictors are con- 
siderably smaller than those of the raw estimates. In addi- 
tion, the set of predictors contains fewer extreme estimates. 
For example, for poststratum groups 34, 39 and 48, the Vos 
estimates of the percent net undercount are 6.04, 0.17 and 
7.51 while the raw estimates are 11.06, -4.14 and 18.76, 
respectively. Most smoothed estimates differ from the direct 
estimate by less than one direct estimated standard error. 
The three largest standardized differences are for Black 
Owner-Large Urban in the West, Black Renter-Large Urban 
in the Northeast, and Non-Black Hispanic Owner-Large 
Urban in the Midwest. In the three cases, the difference bet- 
ween the direct estimate and the smoothed estimate divided 
by the direct standard error is about 1.8. 
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Table 4 
Estimated Percent Net Undercount by Poststratum Group 

Poststratum Group Me s.e. (Y) Vo. S.€. (Vo5) 
Non-Hispanic White Owner Large Urban 

Ie INGE: -2.08 1.04 -0.63 0.60 

2. South 0.69 0.72 0.38 0.44 

3. Midwest -0.26 0.39 -0.13 0.31 

4. West -0.34 0.64 -0.02 0.44 
Non-Hispanic White Owner Other Urban 

SNE: -1.07 0.48 -0.73 0.35 

6. South 0.52 0.43 0.53 0.33 

7. Midwest -0.10 0.40 0.01 0.31 

8. West 0.63 0.58 0.30 0.40 
Non-Hispanic White Owner Non-Urban 

Oy INGE: -0.53 0.69 -0.28 0.47 

10. South 0.18 0.69 0.58 0.45 

11 Midwest -0.70 1.16 0.16 0.64 

12. West 0.29 0.69 0.38 0.46 
Non-Hispanic White Renter Large Urban 

135 NE; 1.17 1.43 2.07 0.61 

14. South 2.62 1.56 8.53 0.64 

15. Midwest 239) 1.70 Pea: 0.60 

16. West 3.28 2 3.10 0.58 
Non-Hispanic White Renter Other Urban 

eee NRE; 3153 1.62 2.29 0.61 

18. South: 3.30 1.86 3.67 0.67 

19. Midwest 1.24 1.13 239 0.53 

20. West 4.70 1.47 3.20 0.57 
Non-Hispanic White Renter Non- Urban 

21) INGE: 6.97 4.67 3.54 =. 0:92 

22. South 6.65 93, 3.60 0.66 

23. Midwest 2.93 1.60 2.36 0.66 

24. West 6.48 2.06 3.48 0.67 
Black Owner Large Urban 

25 IN-E: 1.65 1.96 0.97 0.91 

26. South 2.20 0.94 2.30 0.70 

27. Midwest 0.82 0.88 13 0.67 

28. West 6.49 2.16 2.54 0.96 
Black Owner Other Urban 

290) US: 1.36 1.01 2.05 0.72 
Black Owner Non- Urban 

30 UE US: 3.64 2.03 2.85 0.98 
Black Renter Large Urban 

Slee NIE: 9.13 1.93 Dod, 0.96 

32. South 6.69 PHAN! 6.42 1.10 

33. Midwest 6.38 1.91 5.43 1.03 

34. West 11.06 3:35 6.04 EN? 
Black Renter Other Urban 

3557 U:S: 4.33 1.28 4.99 0.82 
Black Renter Non- Urban 

SOsUES: 4.84 5.95 5.90 1.24 
Non-Black Hispanic Owner Large Urban 

Bie: 0.68 4.44 3.00 1.18 

38. South 259 0.95 Ze 0.72 

39. Midwest -4.14 2.38 0.17 0.97 

40. West 2.98 0.92 2.89 0.68 
Non-Black Hispanic Owner Other Urban 

Alen U:S: 0.95 1.70 2,32 0.87 
Non-Black Hispanic Owner Non-Urban 

42. US. 2.80 2.83 2.88 1.16 
Non-Black Hispanic Renter Large Urban 

43. NE. TDA 4.04 5.85 127 

44. South 10.30 3.11 735 iad) 

45. Midwest 711 3.74 el 1.21 

46. West 6.29 2.09 6.45 0.98 
Non-Black Hispanic Renter Other Urban 

Bile Ups: 7.07 3.10 6.26 1.09 


Non-Black Hispanic Renter Non- Urban 
48. US. 18.76 7.24 7.51 1.38 
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APPENDIX A: Estimation of iy 


The estimators of S; and 65 of ZX, are patterned after 
analysis of variance estimators. The estimation process con- 
tains several steps using improved estimators from one step 
in the next step. We partition the regression problem as 


B, ~ 
+ > 
B, ) 
where (Y,, X,) contains the observations for minorities 
and (Y,, X,) contains the remaining observations. Let Y, 
be an n,-dimensional column vector and Y, be an 


n,-dimensional column vector observations. An initial 
- / LN 7% 
estimator of (B,, B )’ is 


Y 
x 


XO 
0 xX 


1 


Z 


A 


li J homed a NG 
pao ate X)2.Yy 


B, 


where 


is partitioned to conform, to the partition of Y. 
Initial estimators of o and o, are 


é =max{|(¥,-X,B,)’221,(Y,-X,B,)-2,, |e), 0}, 
for i = 1,2, where 
STG rier, y; RA Sar, ~ X Avis): 
82; ~ r{d,, " XA vei)’ Bei ~ XA nai): 


Ay aX xP xt 


and I_, is the n, xn, identity matrix. 
The final estimators are 


6? = max{[(¥,-X,B,)' £4, -X,B,)-2, é:'0}. 


formi— es where 


gpet ieee XA Eoy (ly 9X, Aigo| 
bee (1, XAG, aay, as 
Li > ae sales 


Bp pay le Day Nya ¥ 


1 zzit 
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Estimators of the variance are 
V{6;} 
% 235, 
x tr{] Pr ( Va x, ys sa i -x,A,,,)[ } 
a a 
xtr{{5.,d,,-X Xx, von) o2(1,, -x,Ay,,)[ \ 
for i = 1, 2. The estimated covariance is 
CG. 6} = 2tr {.M,,%,,M,,} 
+ 2d, te {3,, M,, 5M, Ae 
where 
=i eel 
81 T,X Aya) M1, -XAyi) 0 
1 
0’ 0 
and 
0 0’ 
ete eek Ae Set eh 
87 ( ny M22) %:222( n, *2uaa) 
See Searle (1971, Chapter 2 and p. 435). 
APPENDIX B: Approximations for the 
Variance of Predictors 
Our model is 
Y=XBp+wee, (B.1) 


where Y is an n-dimensional column vector, X is annxk 
fixed matrix, 


“J -a((9).( ‘]). 2 


and XZ, is defined in (30) of the text. 


For purpose of variance estimation, we assume ae is an 
unbiased estimator of Z,, distributed as a multiple of a 
Wishart matrix with d, degrees of freedom independent of 
(w, e). We let y be the unknown true vector to be predicted 
and write 


y=XB+w and z=w+e. 
By a Taylor expansion 
§,-y=e-HH; (Y -XB,) 


=e-Hiz+H/X(8,-B)-(A, -H,)z+0, (n by 3.3) 
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where 
=] 
H, = 2,,[9R.+(1-9)Z,,| 
and H = He is defined in (31). The error in B, iS 


“a 


B, -B (B.4) 


=(Xo Xx Ez 


Now = is independent of z and Y - XB is uncorrelated 
with p - B if the true LZ, is used in place of Le 
Therefore 


E{HjX (Bp - B)2'(Hy - Hy)} = (B.5) 


where H, is constructed using Y - XB in the estimators of 
the elements of =e defined in Appendix A and 
H, ==, ps We set ‘the covariance between 6 and H, 
eae to Zero for all @. Now 


fi, - &) [ob, +(1-9)8,] 
-[E,,,, +90, +(1 -@)E,,| ob, +1 -@)¥,,] 
and 
H, -H, =2,,[9(,,-D,,)+(1-9)(2,.-%.)] 


Bip vi 


flee as Cw 1 (D,, a D,,) 


+(1-9)(2,, =2,,)(Be 


3 Lyo| 9(D,. 2D "9 ( 1 * 0)(Zee a. G, 
sd Loew abe 


where Gre =I -H. Thecontribution of D, -D,, to the 
variance of H_ is small relative to the contribution of 
eae ee Therefore, we omit D.. -D,, in our variance 


approximation. Then the expectation 
E{(-Be y=, \e oz 2a, 
(== z\0- Bayt 
a aa G, [2..(tr{A%,, }) rs x. AX,,| G. 2 


where A = Zy Cio as because Zz is independent of S. 
We also omit the term d, "GE ap» a 2k oa aos in our 


ee PP 2z° OP 
variance approximation. 


(B.6) 


The expectation for the term containing (a ae ys 
BAAS (3, = Zipp.) 2p 22 Beh (Sigs 5,.)H,} 
where 
. 1, (6 - 94) 0 
ww = Lay = 


Approximating the expectation by treating z as independent 


of Z,, we obtain 
AsIA(G,) ot Aye Gtouncen 
H; ines H,. (B.7) 
Aj C16, Goda, Anka On 
where 
ALA 
Acl La pork=2) Seo 
Ay, A, “4 
The Taylor expansion of B - B is 
EO. Outs eh Gree 
246. Oi Op qty? 
pol -lyry ls 
H(X DX) ML de) 
5p Map. (03RD. Oe. O92 
ONE, Xie tes 
x (Ge = pe z + Remainder. 
=(X'L, Xo XE 
+ HQ >see) 2h 0 Py 
-L(Z,, - Z yy) Zop2 + Remainder (B.8) 


where Q = X(X’Z,, wx); *X* and GB = (Xe ys 
If %,,, = 2; an “if os is distributed as a multiple of : a 
Wishart with d, degrees of freedom, independent of z, then 


FL (®,,-2,,)%, QE, 22’ Y, QEz, 
x (2, -E,)L'} 
= d,'L[z,tr {219} + Q]L! 


= d," (x'E,)x) (e+). 


Using a similar approximation 


AL(&,,-2,,)2,,XL zz 


» ALS, -2,)EEXLE_Z6,-2,)L'| 


> AL(S,,-2,)e/0 u&.,-2,)L’} 
= d,' (X' EX) | (k+). 
On the basis of this result, we use the approximation 


B-p = (x’E,)x)'x’E,!2-L(&,, -£,,)Ejoz. 
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We assume by is a multiple of a Wishart matrix with 
d,-degrees of freedom and approximate Loo meee with 
(1 -@) (. -Li,,,). We have 

a-oFAlL(&, ©.) 29 p02 


2“ ate -z,,) L' } 


= (1-9)'d, 'L 2, ett ae z,,| 


+k, bai 2 SallA (B.9) 


The dominant term is that associated with the trace and we 
retain only that term in our approximation. Thus, an approx- 
imation to the variance of B is 


V_p=LE,, L’ 


+d," (1-9)tr{Z, 2, Zp, }LE,,L! 


(B.10) 


Combining results (B.6), (B.7), and (B.9), a crude 
estimator of the variance of the predictor (31) is 


V{¥}- Hy H+ GEG, 
+H, XV, XH 6 +144 +133, (B 11) 
where 
H,=1-G,, 


X tr Boy 2, Dipn Bee} Lig Bee(1 +8,)L/, 


‘9 ‘99 ee} *~p 
=e, 1Q@-l}w, cal 1acl 

hats $x’) X'So9 > 

A -1 re A A =i 

Soe i (Zoo ee Xoo) 9 

ae >. 6 
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f % i, Ai, C{6;, 53} 
Y,, =H, xc e — H,; 
A,,C{6,,6;}  A,V {63} 
© Ay, Ay Cee 
A= “k A = Zio U.. Moy? 
Ay 22 
Vis ay j=1,2, is the estimated variance of 3°, and 


C{6;, 65} is the estimated covariance between 6 and 6}. 
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See Appendix A. The estimator of the variance of B 
contains an adjustment for the fact that (X’ Si wXOF Las a 
biased estimator of (X’E_)X) 7, 
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Census Coverage Error: A Demographic Evaluation 


REJEAN LACHAPELLE and DON KERR! 


ABSTRACT 


The 1996 Canadian Census is adjusted for coverage error as estimated primarily through the Reverse Record Check (RRC). 
In this paper, we will show how there is a wealth of additional information from the 1996 Reverse Record Check of direct 
value to population estimation. Beyond its ability to estimate coverage error, it is possible to extend the Reverse Record 
Check classification results to obtain an alternative estimate of demographic growth — potentially decomposed by 
component. This added feature of the Reverse Record Check provides promise in the evaluation of estimated census 
coverage error as well as insight as to possible problems in the estimation of selected components in the population 


estimates program. 


KEY WORDS: Census coverage error; Population estimates; Reverse record check. 


1. INTRODUCTION 


The Reverse Record Check (RRC), in various forms, has 
been used by Statistics Canada since the 1960’s to estimate 
coverage error in the Canadian Census (Fellegi 1969; 
Brackstone and Gosselin 1973; Gosselin 1976; Burgess 
1988; Carter 1990; Royce, Germain, Julien, Dick, Switzer 
and Allard 1994, Statistics Canada 1999). Using the 
Reverse Record Check, Statistics Canada has produced a 
long time series of population estimates, from 1971 through 
to the present, fully adjusted for census undercount. The 
current paper will demonstrate how there is additional 
information in the Reverse Record Check, which from a 
demographic perspective, can be exploited for the purposes 
of population estimation. 

The demographic statistics program at Statistics Canada 
uses information from vital statistics, the most recent cen- 
sus, and various administrative sources in generating highly 
accurate and up to date population estimates. Information 
on births, deaths, immigration, emigration, among other 
demographic components, can be used to estimate popula- 
tion growth since the previous census. With each quin- 
quennial census, a cycle ends and the accuracy of these esti- 
mates are put to the test (Romaniuc 1988). Systematic com- 
parisons can be made between these estimates of growth 
and estimated growth as implied by comparing subsequent 
censuses (after adjustment for census coverage error). 

The resultant difference (conventionally referred to as 
the error of closure of the intercensal population estimates) 
has a far from obvious interpretation. While a large error of 
closure is suggestive of problems in the population esti- 
mates, its specific nature is far from obvious (as to which 
demographic components are specifically responsible for 
the error). Furthermore, a honest appraisal of this closure 
error might suggest not only problems in the population 
estimates, but also potential problems in census coverage 


studies themselves (at the beginning and/or end of the 
intercensal period). 

The current paper will demonstrate how an alternative 
estimate of demographic growth is possible, as based expli- 
citly on the RRC classification results. Additional infor- 
mation is available, which assists greatly in the interpre- 
tation and decomposition of this closure error. Three alter- 
native estimates of demographic growth for the intercensal 
period will be presented in the following section, including 
growth as estimated as part of the regular program of popu- 
lation estimates, implicit growth as obtained in comparing 
consecutive censuses, and growth as based explicitly on 
RRC classification results. Section 3 demonstrates how this 
RRC based estimate of growth can assist in the decompo- 
sition and interpretation of closure error, providing evi- 
dence of (i) bias in selected components of the population 
estimates, and (11) possible problems in the RRC results. 
Section 4 presents the results from this decomposition, 
followed by a brief discussion of its implications for both 
census coverage error measurement and the population 
estimates program. 


2. ALTERNATIVE ESTIMATES OF 
DEMOGRAPHIC GROWTH 


2.1. Administrative Record Based Estimates of 
Growth: Post-Censal Estimates 


Statistics Canada’s regular program of population esti- 
mates involves the continuous registration and estimation of 
demographic events, as based on vital statistics and various 
administrative data sets. These events are added or sub- 
tracted from the population documented in the previous 
census (component method). In estimating a province’s 
population on Census day 1996 (?,...4,): 


Réjean Lachapelle, Demography Division, Main Building, Tunney’s Pasture, Statistics Canada, Ottawa, Ontario, K1A 0T6; Don Kerr, Department of 


Sociology, University of Western Ontario, London, Ontario, N6A 5C2. 
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D i 


91-96 = 


= = + 
P.s96 = P91 * Bo1-96 ~ Po1-96 * 91-96 


ANPRg, 9¢*NMo, 96 (1) 


91-96 

The baseline population (P,, ) for this estimate builds on 
the 1991 Census after adjustment for all forms of coverage 
error, including net census undercount as measured through 
the 1991 RRC. The postcensal estimate can be obtained by 
adding or subtracting from this baseline the number of 
births between censuses (Bo, _9¢); the number of deaths 
(Dy; -9¢)» immigrants (Jy, o.), emigrants ( Foicog) se DE 
interprovincial migration (NM,,_,,), and the net gain or 
loss of nonpermanent residents (4A NPR,, _9¢)- 

Non-permanent residents (NPRs) are persons with legal 
temporary status in Canada (e.g., persons holding student or 
employment authorizations, minister’s permits, refugee 
claimants, as well as their non-Canadian born dependents). 
Unlike with interprovincial migration, net gain or net loss 
of NPRs is not estimated through “flow” data on the 
ongoing in and out-flows of non-permanent residents, but 
rather estimated by comparing over time “stock” data on the 
total number of non-permanent residents living in the 
country. Further details of methodology, data sources and 
data quality issues can be obtained from the quarterly and 
annual releases of the population estimates program 
(Statistics Canada 1999; 2000). 


2.2. Implicit Estimate of Growth 


An implicit estimate of growth can be derived using the 
1991 and 1996 Censuses, with both censuses adjusted for 
net undercount. With the exception of a small number of 
refusal Indian reserves, whose population figures are esti- 
mated independently, gross undercoverage was estimated 
entirely through the RRC in 1996, whereas gross overcove- 
rage was a combined estimate from three studies (the RRC, 
the Collective Dwellings Study and the Automated Match 
Study). In 1991, the RRC was used only in the estimation 
of gross undercoverage, whereas gross overcoverage was 
estimated through a smaller study, the Private Dwelling 
Study, in combination with the 1991 Collective Dwelling 
and Automated Match studies. In addition, persons missed 
on refusal Indian reserves were estimated as part of the 
1991 Reverse Record Check. 

In the early evaluation of the 1996 coverage studies, the 
implicit growth obtained with the above adjustments was 
considered unrealistic. It has since been established that 
part of the 1991 estimate of net undercount was in error, 
and would have in reality been lower had selected methodo- 
logical enhancements been introduced as in 1996 
(Tourigny, Clark and Provost 1998). It has been shown that 
(i) a number of persons initially classified as missed in 1991 
was too high due to misclassification, and (ii) the 1991 
estimate of “overcount” was too low. As a result, 1991 
estimates of undercount and overcount have been revised to 
reflect the impact of these methodological changes 
(*"U,,, “’Oy, ). In addition, for reasons of consistency with 


1996, separate modeled estimates of refusal Indian reserves 
(independent of the RRC) have been added to the Census in 
1991. 

More specifically, implicit growth (a!) is obtained as: 


ie : 
poieeligge he k= 
ve p67 oe Oca ~IRercoe} > 


{Poi + U5 - Op, + IR IRexcoi} (2) 
where final population figures (P,,, Py, ) are obtained using 
previously published census figures (P5,, Pg, ) adjusted for 
undercoverage (U,,, ““U,,) and gross overcoverage 
(Og,, “Og, )- In adding independently modeled estimates 
of refusal Indian reserves (IR,,,,, IR5,,,), it is necessary to 
remove that portion of the RRC estimate of gross under- 
coverage that corresponds to these reserves 
(TRercog: Rerco): The results presented in the current 
paper take these changes into consideration. 


2.3.1. RRC Based Estimates of Growth 


The Reverse Record Check (RRC) is a record linkage 
and matching procedure that attempts to trace all persons in 
its sample, interview them to obtain a census day address, 
and match their records to individual census documents. 
This involves the construction of a sample intended to 
represent the same target population as the census being 
evaluated. This sampling frame, obtained in a manner that 
is totally independent of the census being evaluated, is 
constructed using the previous census, birth registrations 
over the intercensal period, administrative lists of inter- 
censal immigrants, and an up-to-date listings of non- 
permanent residents. Persons missed in the previous census 
are represented by a sample of cases classified as “missed” 
in the previous RRC, in the absence of a complete list of 
such persons. 

By working with this sample, the RRC targets all persons 
who could have potentially been part of the 1996 Census 
universe. Except for a very small sub-population of retur- 
ning emigrants (Canadian citizens and landed immigrants 
who were abroad during the previous census), the RRC 
sample is complete and fully representative. The subsequent 
classification (missed, enumerated, emigrated, abroad, 
deceased or out of scope) is applied in the estimation of 
“missed” in the current census. At the same time, this 
classification also holds the potential for further inferences, 
i.e., an additional estimate of demographic growth for the 
intercensal period. 

To estimate demographic growth using the RRC, it is 
useful to consider the following two equations. In the first 
equation, the target population of the 1991 Census (P51) iS 
expressed in terms of all potential classification outcomes in 
1996. In the second equation, it is possible to move in the 
opposite direction — by expressing the 1996 census target 
population (Poo) in terms of all possible statuses in 1991 (or 
in the case of births and immigrants, the intercensal period). 
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where: 
°'PP,,- Canadian citizens and landed immigrants in 
Canada in 1991, also targeted by the 1996 
census 

NPRs in Canada in 1991, also targeted by the 
1996 census as NPRs 

NPRs in Canada in 1991 who became landed 
immigrants over the intercensal period 
Canadian citizens and landed immigrants in 
Canada in 1991, who died over the inter- 
censal period 

NPRs in Canada in 1991, who died over the 
intercensal period 

persons with the right to live permanently in 
Canada (citizens and landed immigrants) that 
are not in the designated census target popu- 
lation 

Canadian citizens and landed immigrants in 
Canada in 1991, who are outside the 1996 
Census target population 

NPRs in Canada in 1991, who became 
landed immigrants or citizens, and are 
outside the 1996 census target population 
persons who have never been citizens or 
landed immigrants, and are not in the 
designated census target population 


NPRs in Canada in 1991, who did not 
become landed immigrants, and are outside 
the 1996 census target population 

births over the 1991-1996 period, and in the 
1996 census target population 

persons not in Canada in 1991, who arrived 
over the intercensal period, and are NPRs in 
the 1996 census target population 
immigrants who landed over the intercensal 
period, and are in the 1996 Census target 
population 
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gepp ~ returning emigrants, i.e., Canadian citizens 


and landed immigrants outside the census 
universe in 1991, and in the 1996 Census 
universe 


An estimate of growth (a®®°) can be obtained by 
subtracting the former equation from the latter: 
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With the previously introduced sampling frames and 
classification outcomes, all terms (with the exception of the 
last term: returning emigrants) can be directly estimated 
from the 1996 RRC itself. The census target population in 
1991 can be approximated through the sample drawn from 
the census and missed frames — with the identification of 
relevant classification outcomes. The census target popu- 
lation in 1996 can be approximated through all persons 
classified as either enumerated or missed in 1996. The final 
term (7.e., returning emigrants) can be obtained independent 
of the RRC using the 1996 Census 5-year mobility variable, 
in identifying all persons outside the country five years ago 
(excluding recent immigrants and NPRs). It is possible to 
express this same RRC based estimate of demographic 
growth at the provincial level by incorporating an estimate 
of interprovincial migration. As the RRC relied on Health 
Care Files in Canada’s two northern territories (the Yukon 
and NWT) with administrative lists of addresses current to 
census being evaluated, this estimate of growth is not 
possible for the relatively small populations living in 
Canada’s far north. 

A minor problem in the RRC design persists that poten- 
tially introduces a slight bias into its classification results. 
Unfortunately it is not possible to identify all NPRs in the 
RRC sample, with the potential for an unknown amount of 
frame overlap (i.e., between the census, NPR and 
immigrant frames). As NPRs in the census can only be 
identified through the census long form (which is distri- 
buted to about 20% of all households), it is possible that 
some NPRs living in Canada in 1991, selected in the census 
frame, were also selected in either the immigrant or NPR 
frames (without being identified as such). While the RRC 
attempts to adjust for this overlap by identifying all such 
persons in the immigrant and NPR frames, an unknown bias 
exists to the extent that this is unsuccessful. This difficulty 
in identifying overlap leaves the potential of too many 
immigrants and/or NPRs in the sample, or too few, if too 
many persons are removed from the aforementioned frames. 
The latter outcome can subsequently deflate the estimate of 
demographic growth, gross undercoverage (among other 
classification outcomes), whereas the former has the 
Opposite outcome. 


RRC based Estimate of Growth: A More 
Detailed Decomposition 


coe 


While both the postcensal and RRC based estimates of 
demographic growth should be highly comparable, the 
specific terms within each are not meant to be directly 
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equivalent. For example, births in the postcensal estimates 
denote all intercensal births occurring to a population — 
irrespective of whether such births move or die — whereas 
births in the discrete equation denote all births occurring yet 
still in Canada at the end of the intercensal period. With 
this in mind, it is possible to expand on the RRC based 
equation, to derive terms that are more comparable to those 
used in the postcensal estimates. The RRC based estimate 
of demographic growth can then be used in the evaluation 
of the components of demographic growth that enter into 
the component method. 

To expand on this equation, it is useful to begin with 
births, again expressed in terms of possible RRC classifi- 
cation outcomes. As previously indicated, the birth term as 
included in equation (5) is only part of all births occurring 
over the intercensal period. More comprehensively, all 
births can be expressed as: 


91-96 _ 91-96 91-96B 91-96B 
B - Bog * Dyg + Eo6ER (6) 
where: 
B°!-°6 = all intercensal births 
°1°9°B = _ all intercensal births ultimately classified as 
either enumerated or missed in 1996 
91-96BT) = deaths of intercensal births 


96 
ele toe persons outside target population in 1996 yet 
born in Canada over the intercensal period 


Similarly, all immigrants can be expressed as: 


91-96 _ 9IEX 9INP 91-961 91-961 
I 5 Togpp + Cogpp * Dyg + Eyerr (7) 


where: 

aie fi intercensal immigrants ultimately 
classified as either enumerated or missed 
in 1996 

°INPCcpp = _all NPRs in 1991 who obtain landed 
immigrant status and are ultimately 
classified as either enumerated or missed 
in 1996 

*1-98ID,, = deaths occurring to landed immigrants 
over the intercensal period 

71°F cep = =~ EMigrants among intercensal immigrants 


(irrespective of whether or not they were 
living in Canada as NPRs in 1991) 


In combining equations 5, 6 and 7, demographic growth 
can be restated as: 
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Given that the final term of (8) is equivalent to: 
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This expanded version of equation (5) provides a break- 
down of demographic growth at the national level, and 
allows for more meaningful comparisons with components 
estimated through administrative records. All terms, except 
for °'®RE,.., and NP,, can be obtained directly from the 
1996 RRC. The aforementioned hole in the RRC sampling 
frame requires an independent estimate of returning 
emigrants whereas the nature of the sample frame for NPRs 
explains the absence of the latter term. Rather than a listing 
of all NPRs to enter Canada over the intercensal period (as 
was the case with immigrants), the RRC relies on the most 
up to date administrative listing of NPRs in the 
establishment of its sampling frame (with no information on 
the number of NPRs living in Canada in 1991). 

Postcensal estimates document demographic growth 
through the “continuous” registration and estimation of 
demographic events over time. The RRC estimates growth 
via information on the status of individuals as identified on 
at least two “discrete” dates (at the beginning and end of the 
intercensal period). Irrespective of this minor conceptual 
distinction between “continuous” versus “discrete” estima- 
tion, each term of equation 11 (within each set of paren- 
thesis) roughly corresponds to a separate component as 
documented using administrative records. The first term 
identifies all intercensal births (i.e., the weighted sum of the 
birth frame), the second term includes deaths (classification 
results across the birth frame, the missed frame, the census 
frame and immigrant frame), the third term includes all 
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immigrants (7.e., the weighted sum of the immigrant frame), 
the fourth term includes emigrants (classification results 
across the birth frame, the immigrant frame, the missed 
frame and census frames, as well as the returning emigrant 
component), and the fifth term corresponds to net gain or 
loss of NPRs. As the number of NPRs living in Canada in 
1991 is not available in the 1996 RRC, for current 
purposes, this latter term is obtained using the 1991 census 
count, after adjustments for undercoverage. Again, it is 
possible to express this equation at the provincial level. 

With equation (11), a detailed evaluation of the post- 
censal estimation program is possible. For example, if 
differences persist between RRC based estimates and post- 
censal estimates, it is possible to determine how much of 
the difference in estimated growth can be traced back to 
differences in migration (typically estimated with some 
difficulty in the postcensal estimates program) and how 
much can be traced to differences in natural increase. 
Briefly, Table 1 includes all of the aforementioned esti- 
mates of growth, including implicit growth, the growth as 
based on administrative records, and the two alternate esti- 
mates of growth as based on the RRC (simplified and ex- 
panded equations). Slight differences exist between the 
simplified and expanded equations — yet not nearly of the 
same size as with the other estimates (implicit, postcensal). 
In explanation of the differences between the two RRC 
based estimates, the simplified equation does not require the 
same detailed classification as with the expanded equation, 
is not biased to the same extent by the aforementioned 
problem of frame overlap, and does not rely on the 1991 
census count of NPRs. The differences observed with the 
remaining estimates are the focus of the current 
decomposition. 


Table 1 
Alternate Estimates of Growth, 1991-1996, Canada and 
Provinces/Territories 
Implicit Population RRC RRC 
Growth Estimates simplified expanded 
Administrative 
records 

NFLD. -17,997 -9,263 -17,897 -17,751 
PEI. 5,404 5,483 2,568 1,583 
N.S. 15,781 24,271 17,075 16,860 
N.B. 7,714 13,097 12,017 11,276 
QUE. 206,307 300,849 261,357 252,014 
ONT. 659,349 766,568 668,443 655,572 
MAN. 23,682 24,981 WeSTT 6,288 
SASK. 15,953 11,098 11,524 9,312 
ALTA 186,594 186,986 151,944 159,907 
B.C. 505,025 466,611 465,864 472,342 
YUKON 3,085 2,329 N/A N/A 
N.W.T. 6,837 5,864 N/A N/A 
Provinces 
(excl terr) 1,607,771 1,790,681 1,580,273 1,567,404 
Canada 1,617,693 1,798,874 N/A N/A 
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3. A DECOMPOSITION OF CLOSURE ERROR 


Implicit growth for the 1991-96 period is obtained only 
after all adjustments have been made to the censuses for 
coverage error, including revised 1991 figures on gross 
undercount and overcount and refinements for refusal 
Indian reserves. Alternatively, the RRC based estimate of 
growth (simplified version) is obtained in working with 
approximations of the 1991 and 1996 target populations, 
i.e., the census and missed frames of the 1996 RRC and all 
persons classified as either missed or enumerated in this 
study. For this reason, there are minor differences between 
the two estimates that need to be more clearly identified in 
a full decomposition of closure error. In this context, it is 
useful to express implicit growth obtained with final popu- 
lation figures in terms of these approximations (sampling 
frames and classification outcomes). In a similar manner, 
as the error of closure is the difference between implicit 
growth and the growth associated with the postcensal esti- 
mates, the error of closure can also be expressed in terms of 
these approximations. 

To simplify the presentation, let 6 represent all possible 
negative growth terms in equation (5) and 7 as all possible 
positive growth terms: 


i) = 


91PP 91NP 91PP 91NP 91NP (12) 
Ce Dygt Dot Eccrrt — Eoeert+ — Ea6Ex) 


7 790206 91EX 91EX 91FR 
n=O Bygt  "logpp + Logup + RE gepp) (13) 


The population enumerated in both censuses can be 
represented as: 


pie ape HNP. igsacah (14) 


Since the final population figures (P,,, Po.) used in the 
estimation of implicit growth involve separate modeled 
estimates for refusal Indian reserves, it is useful to restate 
the RRC based estimate of growth after specifically 
delineating such reserves. In designating persons living in 
refusal reserves in 1996 that were in the target population 
in 1991 as °'ZR,., the growth of these reserves through 
either migration or birth as estimated by the RRC by np, 
and redefining *!P,, to exclude all persons associated with 
these two terms, it is possible to return to equations (3)-(5) 
as: 


ay 
Py, Se Pio AIR 20 (15) 


T 
Pog = Pog + Rog *Tyg * 0 (16) 
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In expressing implicit growth in terms of the RRC 
sampling frames and classification outcomes, it is useful to 
build on the RRC estimate of growth (equation 17) in 
defining total growth beginning with P,, rather than Psi. 
In recognition that the final population estimate -(Po4) iS 
equivalent to the census and missed frames (Po) minus 
overcoverage (‘°O,,) plus refinements for refusal Indian 


reserves. (R54; — Rapes) it follows: 


Doe - Poy =N + Mp -5 + “Oy, + TRerco: ~/Roim)- (18) 


On the other hand, this target population (Pos) can also be 
expressed as: 


Pog = EN gg + Ugg + RE gepp (19) 
where EN,, is an estimate of the number of persons 


enumerated in 1996. In recalling from equation 2 that: 


Poats B 96 * Uy6 ~ O96 + Roem ~ Rercos) (20) 


implicit growth (a! ) can be expressed in terms of the RRC 
based estimates of growth, as: 


ir 
al = Pye — Py, = (Po¢ Poe) 


{(1-8)} + {Mp ~ Roy Rercos) + Rog /Rercos)} * 


+ (Po6~ Po) = 


{(Po6 ~EN gg OIFERE yepp + "Op ~Oy)}- ey 

Implicit growth (a!) can be defined as the sum of (i) a 
RRC based estimate of growth (excluding refusal Indian 
reserves), (ii) a second term depending on the decision to 
estimate the refusal Indian reserves by an independent 
model, and (iii) a third term that involves a comparison of 
the RRC based estimate of enumerated and the number of 
persons actually enumerated in the 1996 census. 

This latter term (the difference on enumerated) has an 
interesting interpretation, and is considered fundamental to 
the evaluation of the RRC (Tourigny, Bureau and Clark 
1998; Royce 1993). Significant differences with this term 
can be read as implying either sampling errors and/or 
possible biases, as either classification error and/or problems 
in sample selection. To make this comparison meaningful, 
1996 overcoverage and an estimate of returning emigrants 
are removed from the census counts - as neither can be 
included in the estimate of enumerated. Similarly, since the 
RRC selects part of its sample from the previous census, it 
inevitably carries forward some overcoverage inherent in its 
weights — which must subsequently be removed from its 
estimate of enumerated. These adjustments are included in 
the third term (the third set of brackets) in equation 21. 

While the estimate of enumerated is inflated by the 
weights associated with overcoverage in the 1991 Census 


frame, only a portion is directly associated with this 
estimate — with the remainder spread across the other classi- 
fication outcomes. Consequently, all classification results 
in the aforementioned equations are also slightly overstated. 
For the purposes of the current decomposition, this minor 
distinction is ignored. This is another reason, albeit of 
minor impact, why the RRC-based estimate of growth is 
different from the implicit estimate, as the latter is not 
biased by this overcoverage. 
From the above, the error of closure is equivalent to: 


D Rb. 
491-96 491-96 ~ 
[451 -96-{(n-8)} - 
{Mp ~URoim Repco) * Roem /Rercos) }] = 
bee 1FR Re 


C rev 22 
[{ (Po6-EN, g6ppt Io; ~ oe) }]- (22) 


In the decomposition of closure error, the first term 
inside brackets [ ] highlights the difference between the 
postcensal estimate of growth and the combined RRC esti- 
mate of growth (including refusal reserves, after refine- 
ments for modeled estimates). The second term (the 
difference on enumerated) provides evidence as to possible 
difficulties in the coverage studies. Theoretically, with the 
absence of sampling and non-sampling error in the RRC, 
this latter term should be negligible. 


4.1. Decomposition Results: Closure Error 


Table 2 presents closure error after finalizing both the 
1991 and 1996 estimates of population. By adding net 
undercount to the 1996 published Census figures, along 
with independent estimates of refusal Indian reserves, 
Canada’s 1996 Census day population, adjusted for cover- 
age error is estimated at 29,619,539. This figure is appre- 
ciably lower than the Census day estimate as generated 
through the postcensal estimates program of 29,800,720. 
The difference between the two figures — which is equiva- 
lent to the aforementioned difference between implicit 
growth and growth as based on administrative records — 
was higher than anticipated given past experience, at 
181,181 (or .61% of the 1991 Census Day population). 

Across provinces/territories, closure error is found to be 
particularly pronounced in Newfoundland (1.56%), in 
Canada’s north (at -2.38% in Yukon and -1.44% in the 
NWT), and somewhat surprisingly, in its three largest 
provinces (as 1.30% in Quebec, .97% in Ontario and -.99% 
in British Columbia). Regionally, closure errors larger 
than the national average are observed across eastern and 
central Canada (except for P.E.I.) while the western 
provinces have closure errors lower than the national one. 
It is specifically these errors that the current decomposition 
seek to evaluate and explain. 

Table 3 presents the results from this decomposition, 
with closure error decomposed into (i) the difference 
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between the estimate of growth based on administrative 
records and the RRC based estimate (simplified version), 
and (ii) the difference on enumerated. Also included is the 
sampling error associated with the RRC estimates. 


4.2. Comparisons between Estimates of Growth 


Across all provinces (with the exception of 
Saskatchewan), growth estimated on the basis of admini- 
strative records is higher than the RRC based estimate. At 


49 


the national level (excluding the territories), this discrep- 
ancy on growth (210,408) appears far more important in 
explaining closure error than the discrepancy on enumer- 
ated (-27,498). While for many provinces the difference on 
growth fell well within expectations in light of sampling 
error, selected provinces require further explanation. For 
example, the difference in growth in Ontario is large 
(98,125), which is almost one half the difference observed 


Table 2 
Coverage Study Results, Relative to Population Estimate (1996 - Census Day) 
{1} {2} {3} {4=1+2+3} {5} {6=5-4} {7=6/4*100} 
1996 census count 1996 net Indian 1996 Census 1996 estimate Error of Error of 
with random undercount Reserves RRC adjusted post-censal (i) closure closure (%) 
additions 

NFLD. Doleio2 9,424 0 561,216 569,950 8,734 1.56 
P.E.L. 134,557 1,149 iS) 135,881 135,960 JS 0.06 
N.S. 909,282 20,821 0 930,103 938,593 8,490 0.91 
N.B. 738,133 14,225 518 752,876 198,299 5,383 0.71 
QUE. 7,138,795 116,750 12,427 7,267,972 7,362,514 94,542 1.30 
ONT. LOTS33573 301,368 20,849 11,075,790 11,183,050 107,260 0.97 
MAN. 1,113,898 18,881 B15 1,133,094 1,134,393 1,299 0.11 
SASK. 990,237 28,051 586 1,018,874 1,014,019 -4,855 -0.48 
ALTA. 2,696,826 66,327 11,287 2,774,440 2,774,832 392 0.01 
B.C. 3,724,500 142,443 3,136 3,870,079 3,831,665 -38,414 -0.99 
YUKON 30,766 1,022 0 31,788 31,032 -756 -2.38 
N.W.T. 64,402 3,024 0) 67,426 66,453 -973 -1.44 
Canada 28,846,761 723,485 49,293 29,619,539 29,800,720 181,181 0.61 


(i) Post-Censal Estimates for May 14", obtained with final components for intercensal estimates. 
Final Estimates (Sept. 24", 1998) of Net Undercount, 1991 an 1996. 


Table 3 
Decomposition of Closure Error 
Province/Territory Error of Difference between Difference on SBSor 
Closure Dem. and RRC SE of enumerated estimates 
Estimates of Growth _ estimates 

NFLD. 8,734 8,634 4,889 100 5,176 
PEL 79 2,915 2,425 -2,836 2,462 
N.S. 8,490 7,196 9,011 1,294 9,455 
N.B. 5,383 1,080 (Es 4,303 7,918 
QUE. 94,542 39,492 25,493 55,050 29,310 
ONT. 107,260 98,125 41,212 9,135 51,300 
MAN. 1,299 17,604 10,108 -16,305 10,370 
SASK. -4,855 -426 9,187 -4,429 10,200 
ALTA. 392 35,042 19,067 -34,650 21,618 
B.C. -38,414 747 20,518 -39,161 22,996 
YUKON -756 N/A N/A -108 270 
N.W.T. -973 N/A N/A -284 464 
Canada without 182,910 210,408 43,951 -27,498 58,724 
Territories 
Canada 181,181 N/A N/A -27,890 58,762 
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Table 4 
Estimated Components (1991-1996) as Compiled by Demography Division and RRC Discrete (detailed) Measurement 
NFLD PEI NS NB QUE ONT MAN SASK ALB BC CANADA 
(without terr) 
Births 
Demography 31,748 8,803 55,994 44444 453,556 730,520 81,485 70,382 199,484 229,911 1,905,927, 
RRC SNE) 8,782 55,984 44,444 454,332 729,744 81,485 70,382 199,484 229 SI 1,905,927 
Difference -3] 22 10 0 -716 776 0 0 0 0 0 
Deaths 
Demography -19,286 -5,692 -37,677 -28,567 -252,628 -376,760 -45,858 -40,652 -75,798 -126,935 -1,009,853 
RRC -18,530 -6,913 -43,820 -29,354 -273,617 -400,047 -56,108 -40,143 -74,640 -138,433 -1,081,605 
Difference -756 1,221 6,143 787 20,989 23,287 10,250 -509 -1,158 11,498 UTS2 
Immigration 
Demography 3,411 TTB. 14,489 3399 189,905 618,869 22,004 11,282 84,130 213,506 1,161,726 
RRC 3,538 820 14,058 3,614 189,905 ONG TAD PPE) 11,157 = 84,130 216,892 1,165,113 
Difference -127 -49 431 -255 0 -1 -125 125 0 -3,386 -3,387 
Emigration 
Demography -671 -206 PING! -2,429 -15,490 -48,609 -5,684 -2,493 -19,718 -17,834  -115,431 
RRC -2,227 -455 -7334 -3,889  -55,766 -168,556 -10,871 -7,133 -33,689 -31,739 -321,659 
Difference 1,556 249 5,037 1,460 40,276 119,947 5,187 4,640 13,971 13,905 206,228 
Interprovincial 
Migration 
Demography -23,074 1,643 -5,288  -3,255 -51,176 -40,850  -25,336 -26,644 7,155 167,809 984 
RRC -32,767 -886 -1,479 — -2,933 -49,395 -37,505  -29,765 -25,095 -10,321 191,222 1,076 
Difference 9,693: 2,529 -3,809 -322 -1,781 -3,345 4,429 -1,549 17,476 -23,413 -92 
Non-permanent 
Residents 
Demography -1,406 164 -950 -455 -23,353 -116,602 -1630 -777 -8,267 554 = -152,722 
RRC 455 236 -549 -606 -13,445 -86,934 -582 144 -5,057 4,890 = -101,448 
Difference -1,861 -72 -401 Tor -9,908 -29,668 -1,048 -921 -3,210 -4,336 -51,274 
Total 
Demography -9,263 5,483 24,271 13,097 300,849 766,568 24,981 11,098 186,986 466,611 1,790,681 
RRC -17,751 1,583 16,860 11,276 252,014 655,572 6,288 9312 159/907 472,343 1,567,404 
Difference 8,488 3,900 7,411 1,821 48,835 110,996 18,693 1780" 27,079 -5,731 223,277 


nationally. Similarly, Newfoundland, Quebec, Alberta and 
Manitoba, together explain a large part of this difference. 
In providing some indication as to the factors responsible 
for these differences, Table 4 presents comparisons using 
equation 11 (detailed equation). Alternative estimates are 
provided on (i) births, (11) deaths, (111) immigration, (iv) 
emigration, (v) interprovincial migration and (vi) net 
change in the number of non-permanent residents. The 
most important problems in the explanation of closure error 
are obvious in Table 4, with specific reference to emigra- 
tion. As Canada does not have a complete border registra- 
tion system, emigration is clearly the weakest of all the 
components to enter into the population estimate program. 
Without access to direct information on the number of 
persons leaving Canada, the RRC, with its exhaustive 
tracing, record linkage and direct interviewing procedures, 
is considered an improvement over any other data sources 
currently available. Although there are known problems in 
the RRC (for example, the previously mentioned frame 
overlap), the current evaluation points to an obvious error 
in the postcensal estimates, i.e., an understatement of 
population outflow from Canada. Overall, the difference as 
observed nationally (206,228) explains the bulk of the 
closure error documented in 1996. Similarly with Ontario, 
difficulties in the estimation of emigration appear to be 
fundamental (with a difference of fully 119,947). 


Without being decisive, the current decomposition also 
suggests other problematic components beyond emigration 
in the explanation of closure error for specific provinces. 
For example, the results suggest that estimates of inter- 
provincial migration might be somewhat misstated for 
British Columbia and Newfoundland (after acknowledging 
the differences observed on these components and corre- 
sponding closure errors). Overall, an acceptance of the 
RRC on these more difficult to estimate migratory flows — 
would not only explain the largest part of this difference in 
growth — but also the largest part of 1996 closure error. 
With the closure error that remains, it is useful to turn to the 
observed difference on enumerated. In so doing, the 
emphasis shifts away from potential problems in the 
postcensal estimates. 


4.3. Comparisons between Estimates of 
Enumerated 


While the difference in enumerated observed nationally 
is much smaller than the difference documented on growth, 
for about half the provinces, this difference is of compa- 
rable if not larger size. In interpreting this fact, it is 
recognized that the RRC was never designed to target the 
“enumerated” population. With the priority of documenting 
the number “missed” in the census, the sampling design of 
the RRC over represents “difficult to enumerate groups” 
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(for example, single young adults), while under 
representing persons easily “enumerated”. Overall, the 
comparison on enumerated bears well for the accuracy of 
the RRC — with non-significant differences across all 
provinces/territories. Nevertheless, the differences observed 
in a few provinces are reason for concern, being very close 
to statistical signifi.cance at the 95% level in Quebec 
(positive difference), and approaching statistical signifi- 
cance in British Columbia, Alberta and Manitoba (negative 
differences). 

In the evaluation of the 1991 coverage study results, two 
alternative hypotheses have been raised in explanation of 
differences observed for the enumerated (Royce 1993). At 
one extreme, it could be argued that all of the difference 
(for a specific province) be explained in terms of the repre- 
sentativeness of the RRC sample, which implies sampling 
error or frame deficiencies of one sort or another. At the 
other extreme, it could be argued that all of the difference 
be explained due to a failure in documenting the true ratio 
of enumerated to other classification outcomes, which 
seems to imply some sort of misclassification error or no 
trace adjustment bias. A correction for the former of the 
two hypotheses has a relatively minor impact on the esti- 
mate of missed (i.e., all classification outcomes are accor- 
dingly inflated or deflated by the proportional difference on 
enumerated). A correction for the latter could have poten- 
tially quite a pronounced impact, as a failure to estimate the 
true ratio implies that all the difference be assigned to other 
categories. 

If the latter hypothesis applies, a correction potentially 
reduces the error of closure in nine out of twelve provinces/ 
territories (i.e., in all provinces under which the error of 
closure is in the same direction as the difference on enume- 
rated). On the other hand, if the differences are due to 
problems in sample representativeness, a subsequent 
correction is expected to have negligible impact, if not 
slightly inflating closure error across most provinces. In 
addition, the evaluation is complicated by the difficulty in 
establishing the comparable census figures. Error can be 
potentially introduced through various sources, including: 
the census-based estimate of returning emigrants 
(OPE RE): too much or too little correction for frame 
overlap, sampling and non-sampling error in the estimation 
of undercoverage in 1991 and 1996, sampling and non- 
sampling error in the estimation of overcoverage, and 
potential error in the classification by province of the enu- 
merated. In this context, further research appears justified 
as to the true character of errors in the RRC estimate of 
enumerated. 


5. CONCLUSION 


In this paper, we have shown how there is additional 
information available through Canada’s census coverage 
measurement program that is of considerable value in 
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population estimation. Beyond the ability to estimate 
census undercount, it is possible to extend the classification 
results from these studies in order to obtain an alternative 
estimate of demographic growth — potentially decomposed 
by component. Using the most important of the coverage 
studies (i.e., the 1996 Reverse Record Check), a new 
method was presented which allows for an independent 
estimate of demographic growth for the intercensal period. 
The Reverse Record Check not only provides what are 
considered highly accurate estimates of census coverage 
error, avoiding some of the correlation biases that have 
hindered post-enumeration studies in other countries, but 
also provides very valuable insight as to the magnitude of 
selected migratory flows of importance to population 
estimation. 

The key to the Reverse Record Check is that it begins 
with a representative sample of all persons who could have 
theoretically been in Canada on census day, with only 
minor deficiencies due to the high quality of vital statistics 
and immigration data in Canada. Through exhaustive 
tracing and interviewing procedures, valuable information 
is then obtained as to the number and characteristics of 
persons successfully enumerated, missed, counted more 
than once, as well as useful information on the numbers 
leaving the country (whether temporarily or permanently), 
the numbers dying, living in another province, and so on. 
With a relatively large sample and considerable expertise 
and effort directed toward minimizing all forms of error, the 
resultant classification results can potentially inform the 
population estimates program. This is particularly true with 
some of the more difficult to estimate migratory flows. 

In planning for the 2001 Census, the goal of minimizing 
all error in the census coverage measurement program 
remains a priority. As these studies have been designed 
with a primary target of estimating the population “missed” 
rather than other classification outcomes (emigrated, 
deceased, efc.), the new demographic approach presented 
in the current paper leads to the logical question, as to 
whether its current design need be reworked somewhat if 
its current usage is broadened. Of interest in this context is 
the fact that these coverage studies appear to provide an 
alternative estimate of growth which rivals that as currently 
available through the population estimates program, and is 
likely superior with respect to selected components. 
Further research about how we might more fully exploit this 
fact appears justified, in improving the quality of the 
population estimates program. 
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Multilevel Modelling of Complex Survey Longitudinal Data 
With Time Varying Random Effects 


MOSHE FEDER, GAD NATHAN and DANNY PFEFFERMANN! 


ABSTRACT 


Longitudinal observations consist of repeated measurements on the same units over a number of occasions, with fixed or 
varying time spells between the occasions. Each vector observation can be viewed therefore as a time series, usually of short 
length. Analyzing the measurements for all the units permits the fitting of low-order time series models, despite the short 
lengths of the individual series. We illustrate this paradigm using simulated data that follow the rotation scheme of the Israel 
Labor Force Survey (LFS). This survey employs a rotating panel sampling scheme of two quarters in the sample, two 
quarters out of the sample and then two quarters in again. The model consists of two-level linear models for single time 
points that are connected by allowing the second level effects (corresponding to households) and the first level residuals 
(corresponding to individuals) to evolve stochastically over time. The likelihood of the model is easily constructed by 
employing the time series properties of the combined model. However, in view of the large number of unknown parameters, 
direct maximization of the likelihood could yield unstable estimators. Therefore, a two-stage procedure is adopted. At the 
first stage, a separate two-level model is fitted for each time point, thus yielding estimators for the fixed effects and the 
variances. At the second stage, the time series likelihood is maximized only with respect to the time series model parameters. 
This two-stage procedure has the further advantage of permitting appropriate first and second level weighting to account 
for possible informative sampling effects. Empirical results when fitting the model to data collected by the Israel LFS are 
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also presented 


KEY WORDS: Informative sampling; Probability weighted IGLS; Rotating panel schemes; State-space models. 


1. INTRODUCTION 


1.1 Background and Objectives 


In recent years there has been a growing interest in 
fitting models to data collected from longitudinal surveys 
that use complex sampling designs. This interest reflects 
expansion in requirements by policy makers and social 
scientists for in-depth studies of social processes over time, 
rather than of one-time “snap-shots” provided by cross- 
sectional analyses. A familiar example is the estimation of 
gross flows between social and demographic states such as 
employment states or health and education levels. For 
discussions of these issues and the problems they raise with 
respect to the design and analysis of longitudinal surveys, 
see Duncan and Kalton (1987) and Binder (1998). 

Examples of surveys we wish to consider in this paper 
are of three types: 


1. Rotating panel surveys such as labor force surveys 
carried out in many countries. These surveys were 
often designed originally for cross-sectional analysis 
of household and individual data, so as to study labor 
force and other socio-economic characteristics on a 
current basis. Complex rotating sampling schemes 
have later been introduced in order to improve 
comparisons over time. For example, the quarterly 
Israel Labor Force Survey (LFS) employs a rotating 
panel sampling scheme whereby each unit in the 
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sample is interviewed for two consecutive quarters; it 
is left out of the sample for the next two quarters and 
then is interviewed again for two more consecutive 
quarters. In The U.S.A. and Brazil, a more compli- 
cated sampling scheme of 4 months in the sample, 8 
months out of the sample and then 4 months in again 
is used. Australia, Canada and the U.K. employ 
sampling schemes by which sampled units are inter- 
viewed over a succession of months or quarters before 
being dropped from the sample. These kinds of 
surveys are increasingly used for short-term longitu- 
dinal analysis, such as the estimation of gross flows 
between labor force states or studies of social 
mobility. This has not always proved simple due to the 
complexity of the survey designs, difficulties in 
matching and response errors. 


Medium term panel surveys, such as the U.S. Survey 
of Income and Programme Participation (SIPP, 
Herriot and Kasprzyk 1984), the U.S. Panel Study of 
Income Dynamics (PSID, Survey Research Center, 
1984) and the Canadian Survey of Labor and Income 
Dynamics (SLID, Webber 1994). These surveys differ 
from labor force surveys in being specially designed 
for longitudinal analysis of economic and social 
characteristics of households and individuals. For 
example, SIPP includes an intensive investigation in 
the form of a full retrospective interview every 4 
months. It provides a complete work history for the 
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survey period (30-48 months) by combining the 
continuous retrospective four-month recall data with 
a reconciliation of data provided for longer periods. 


3. Longitudinal cohort studies characterized by the 
follow-up of a cohort sample over a long time period. 
For example, in the British Household Panel Survey, 
starting from a sample of addresses selected in 1991, 
data have been collected on the same households in 
subsequent annual waves for over seven years. A wide 
range of data is collected on labor force character- 
istics, economic resources and health and education, 
with emphasis on longitudinal aspects. In this survey 
all members of the originally selected households 
were followed and the sample was supplemented by 
the addition of entrants to the sample households, 
including children born to sample household 
members. Other longitudinal cohort studies such as 
the British National Child Development Study and the 
British Cohort Study have surveyed a cohort of births 
over periods of up to 40 years. See Nathan (1999) for 
description and discussion of the latter three studies. 


Most of the studies associated with these surveys require 
longitudinal analysis for populations that have a complex 
hierarchical structure, based on data collected from com- 
plex sampling designs. Standard analysis of longitudinal 
survey data often fails to account for the complex nature of 
the sampling design such as the use of unequal selection 
probabilities, clustering, post-stratification and other kinds 
of weighting used for the treatment of non-response. The 
effect of sampling on the analysis is due to the fact that the 
models in use typically do not incorporate all the design 
variables determining the sample selection, either because 
there may be too many of them or because they are not of 
substantive interest. However, if the design is “informative” 
in the sense that the outcome variable is correlated with the 
design variables not included in the model, even after 
conditioning on the model covariates, standard estimates of 
the model parameters can be severely biased, leading 
possibly to false inference. Pfeffermann (1993, 1996) 
reviews many examples reported in the literature that 
illustrate the effects of ignoring the sampling process when 
fitting models to survey data and discusses methods that 
have been proposed to deal with this problem. See also the 
book edited by Skinner, Holt, and Smith (1989) and the 
more recent paper by Pfeffermann, Skinner, Goldstein, 
Holmes, and Rasbash (1998) to which we refer in more 
detail below. It should be emphasized that standard 
inference may be biased even when the original sample 
design is simple random within design strata, due to non- 
response, attrition, and imperfect frames that result in 
de facto a posteriori differential inclusion probabilities. 
Special features of longitudinal studies, such as late 
additions of individuals who join panel households, can 
also lead to de facto unequal inclusion probabilities. 


In this paper we propose to deal with the problems 
arising from the hierarchical nature of the target population, 
the longitudinal aspect of the analysis and the effects of 
complex sampling designs by combining three separate 
statistical methodologies. These are multilevel modelling 
(MLM), time series modelling and methods of analysis 
under complex informative sampling. Multilevel models are 
used to deal with the hierarchical structure of many human 
populations like persons within households, pupils within 
classes, classes within schools and so forth. The models, 
extensively employed by social scientists especially in the 
field of education, account for the effects of observed 
covariates at the lower and higher levels of the structure, 
with fixed or random coefficients. Common unobservable 
random effects within the higher levels capture further 
unexplained variations. The method of Iterative 
Generalized Least Squares (IGLS) is commonly used for 
estimating the model parameters, Goldstein (1986, 1995). 

Simple state-space time series models are used to 
combine the multilevel models operating at different time 
points via a set of linear transition equations that account 
for the time series relationships of the random covariate 
coefficients and the higher level random effects. The 
Kalman filter is used for estimating the model parameters 
and predict the random effects for current and future time 
points. Smoothing algorithms can be used for updating past 
predictions, Harvey (1989). Methods of model fitting under 
informative sampling are employed to control the bias 
resulting from the sample selection process. Such methods 
have been investigated in recent years in the context of 
analytic inference from complex sample surveys, mostly for 
cross-sectional analysis of single-level models, cf. Skinner 
et al. (1989). In the present paper we utilize the 
methodology of sample weighting for multilevel modelling 
as developed by Pfeffermann et al. (1998). 

The aims of the present study are then to develop models 
and methods of estimation for longitudinal analysis of hier- 
archically structured data, taking unequal sample selection 
probabilities into account. The main feature of our approach 
is that the model is fitted at the individual level but it 
contains common higher level random effects that change 
stochastically over time. The model enables to predict the 
higher and lower level random effects (like household and 
individual person effects in the present application), using 
the data for all the time points with observations. This 
should enhance model-based inference from complex 
survey data since it permits a better understanding of the 
structure and correlation pattern of the longitudinal mea- 
surements. In particular, it is bound to improve the predic- 
tion of individual measurements compared to the use of 
aggregate time series models, which by their nature fail to 
separate the individual (person) effects from the common 
higher level (household) effects. These advantages are 
partly illustrated in the example of section 6 and more so in 
a related paper by Pfeffermann and Nathan (forthcoming) 
which focuses on the imputation of missing data. It is 
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important to emphasize in this regard that although the 
length of each individual longitudinal record is often very 
short (4 measurements for each individual in our applica- 
tion), the number of records is usually sufficiently large to 
warrant the application of classical time series estimation 
and model diagnostic procedures. In this article we only 
consider parameter estimation under a given model but the 
use of test statistics and diagnostic procedures that employ 
the empirical innovations for model identification follows 
through with minor modifications by virtue of the use of 
maximum likelihood estimation methods and_ the 
consistency of the parameter estimators. 

In section 2 we overview the main features of the afore- 
mentioned statistical methodologies that are employed in 
subsequent sections. In section 3 we propose a model that 
addresses the longitudinal aspects discussed above. Estima- 
tion procedures are discussed in section 4. Section 5 
contains the results of a simulation study carried out for 
assessing the performance of the various estimators under 
different sampling scenarios. Results obtained when fitting 
the model to real data collected by the Israel LFS are 
presented in section 6, followed by a brief summary in 
section 7 of possible model extensions and applications. 


1.2 Literature Review 


Previous work in this area deals mostly with longitudinal 
data in a non-survey context and does not consider hierar- 
chically structured populations. In particular, none of the 
studies that we have come across permits the second level 
effects (common household effects in our application) to 
evolve over time. For example, Goldstein, Healy and 
Rasbash (1994) consider the analysis of repeated measure- 
ments using a two-level model with individuals as second 
levels and the repeated measurements as the first levels. The 
model extends the standard two-level model by permitting 
the first level measurements to be correlated over time. The 
authors consider several possibilities of modelling the auto- 
correlation structure, which include autoregressive models 
when the measurements are taken at equally spaced time 
points and autocorrelation functions when the observations 
are taken at unequal time intervals. In the latter case the 
autocorrelation function is linearized for estimation 
purposes. 

Several authors study the application of time series 
models for the analysis of longitudinal data. In a series of 
papers by Jones and his co-authors (Jones and Ackerson 
1990, Jones and Boadi-Boating 1991, Jones and Vecchia 
1993) and the book by Jones (1993), the authors consider 
observations taken at unequally spaced time gaps. The 
observations referring to the same subject are allowed to be 
serially correlated by postulating continuous autoregressive 
moving average models. These models contain fixed and 
random effects, but do not have a hierarchical population 
structure. Weighted least squares and state space modelling 
combined with the Kalman filter are used for calculating the 
likelihood function. 
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Continuous time autoregressive models for irregularly 
spaced longitudinal data are considered also by Belcher, 
Hampton and Tunnicliffe (1984), using linear stochastic 
differential equations for describing the process generating 
the data. An Empirical Bayes approach is proposed by 
Bryant and Day (1991) for the simultaneous analysis of a 
system of mixed linear models, having linked and serially 
correlated random effects. Chi and Reinsel (1989) consider 
a score test for autocorrelation between individual errors 
under a “conditional independence” random effects model. 
The authors derive a maximum likelihood estimation proce- 
dure and use the estimators for predicting the random 
effects by application of Empirical Bayes. 

Diggle, Liang and Zeger (1994) propose the use of 
generalized linear models for the analysis of longitudinal 
data. They consider a transition (Markov) model by consi- 
dering past values as additional predictor variables. 
Transitional extensions of the GLM are used for maximum 
likelihood estimation under linear link functions, whereas 
for non-linear link functions the estimation is based on 
conditional score functions. Lawless (1999) uses an event 
history approach for the analysis of longitudinal data. By 
this approach, the dependent variable is the number of 
occurrences of a particular event up to a given time point f, 
with the limiting transitional probabilities being modelled 
as functions of the previous history and covariates. 
Zimmerman and Nunez-Anton (1997) propose a structured 
antedependence model for longitudinal data, primarily in 
the context of growth analysis. Neither of the above studies 
considers a hierarchical structure or a complex sampling 
design. 

Finally, Skinner and Holmes (1999) consider a model for 
longitudinal observations that consists of a “permanent” 
random effect at the individual level and autocorrelated 
transitory random effects corresponding to different waves 
of investigation. The authors study two approaches for the 
estimation of the unknown model parameters with both 
approaches accounting for sampling effects and “non 
informative” attritions. The first approach treats the 
repeated observations as correlated multivariate outcomes 
and derives probability-weighted estimators that account for 
the correlation structure. The second approach considers the 
model as a two-level model with “individuals” as the 
second level units and the repeated measurements as first 
level units. Estimation of the unknown parameters under 
this approach is carried out by a modification of the 
PWIGLS method of Pfeffermann ef al. (1998, see section 
D2) 


2. STATISTICAL METHODOLOGIES 
UNDERLYING THE PROPOSED APPROACH 
2.1 Multilevel Models 


In what follows we consider a two-level model for the 
response variable y in a population consisting of 
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i = 1,..., M second level units (household, schools, ...) and 
j=1,..,N, individuals within second level unit 7. The 
model is, 


Yi =xjB +2 ,U, + Zo 8 ys Fal eA of ool ceo, AN al ead) 


where x,,,2;,, and z,,, are known covariate values of 
dimensions p, q and | respectively, B is a fixed parameter 
vector of dimension p and p, ~ N(0, Q) and ey ~ N(0, 6”) 
are independent random second level effects and first level 
residuals of orders p and 1 respectively. 

The inclusion of the multipliers Zo, allows for first level 
heteroscedasticity whereas the common second level effects 
u, explain the (interclass) correlations between individual 
measurements corresponding to the same second level unit. 
In the simple case of the “random intercept model”, 
yy =X i B+u,+ ei» these correlations take the familiar form, 
Corr(y,,, ¥,) = 5, / (6; + 6). The random intercept model 
is often applied for small area estimation (see below). 

As stated in the introduction, models like (2.1) are 
widely used by social scientists for studying the effects of 
the covariate variables and the interrelationships between 
observations corresponding to the same higher level unit. In 
such cases, primary interest is in the estimation of the vector 
coefficient B and the vector 0 of the distinct elements of Q 
and o”. Another, well-known application of the two-level 
model is for “small area estimation”, in which case the 
second levels are geographical areas or other domains of 
study. In small area estimation, the target of the analysis is 
the prediction of the second level (area) means ie B+ Zi U,, 
where X, and Z, are the true area covariate means, and the 
estimation of the model parameters is only an intermediate 
step. See Rao (1999) for a recent review. 

___ Estimation of the unknown model parameters is carried 
out most conveniently by use of the Iterative Generalized 
Least Squares (IGLS) algorithm (Goldstein 1986, 1995). 
For a random sample of m second level units and n, first 
level units within second level unit 7, the model holding for 
the sample data is first written in matrix form as 


y,=X,B +d, t=1...m (222) 


where: “priya, Ve eX [Xj po Xin)’ and d,= 
[dij] with d,, = (z' uy, Hzace, ye Then s-N OV); 
where V, = Z,Q.Z/ + 0° Z = V0); Z,=[z,..z,, }’ and 
Zo; = diag[Zp,, ---29;,]. The IGLS algorithm iterates 
between the estimation of B , with 8 considered known, and 
the estimation of 8, with B considered known. At each 
iteration, the estimate obtained for the other vector para- 
meter on the previous iteration is used as the “known” 
parameter. This process is a special case of the EM 
algorithm and it converges to the corresponding maximum 
likelihood estimators (MLE) under the stated normality 
assumptions. It is known to provide consistent estimators 
under more general conditions. 


2.2 MLM Estimation Under Informative Sampling 


The IGLS algorithm described in section 2.1 assumes 
that the model defined by (2.2) holds for the sample data. 
This would be the case if selection of the first and second 
level units is carried out by simple random sampling. How- 
ever, as discussed in the introduction, the selection of the 
sample could be informative so that the model holding for 
the sample units differs from the model holding in the popu- 
lation. For example, in an educational survey, schools in 
poor areas could be sampled with higher probabilities. In a 
household survey, higher selection probabilities could be 
assigned to households in areas characterized by high 
proportions of minorities or to persons that are unemployed. 
As illustrated by Pfeffermann eft al. (1998) and also in 
section 5 of the present paper, the use of the IGLS algo- 
rithm in such cases could yield severely biased estimators 
for all the parameters. The authors propose therefore a 
probability weighted IGLS (PWIGLS) algorithm that 
protects against informative sampling. 

The algorithm is an adaptation of the pseudo-MLE 
method (Binder 1983, Skinner et al. 1989, Pfeffermann 
1993). Suppose that the two-level model defined by (2.1) 
holds for the target population. Had all the population 
values been observed, the IGLS would converge at the end 
of the iterative process to the census estimators, (B.; 6, ). At 
each iteration, the intermediate estimators (Buy 6.) are 
products of matrices with elements that are functions of 
sums of the population values. When the IGLS is applied to 
sample data, the population sums are substituted by the 
corresponding sample sums. The PWIGLS consists of 
further replacing the unweighted sample sums by weighted 
sums. Denote by 1, =Pr(ies) the second level sample 
inclusion probabilities and by a ii =Pr(jes|ies) the 
conditional first level inclusion probabilities. The PWIGLS 
estimators are obtained by, 1- replacing each second level 
sample sum of the general form )°" | g, by the weighted 
sum "wg, where w, =, and 2- replacing each first 
level sample sum yj-183 bY the weighted sum Lint Mi Sy 
with w,,, = 1, Note that the weighting process requires the 
knowledge of the inclusion probabilities at both stages of 
the selection process and not just the final overall inclusion 
probabilities 7, = T,X T- 

As established by Pfeffermann ef al. (1998), the 
PWIGLS estimators are consistent for the model parameters 
when both the first and second level sample sizes increase, 
but the estimators of the variances are not consistent if the 
first level sample sizes are bounded. For this case, the 
authors propose appropriate scaling of the weights w ,, that 
eliminates the bias, provided that the sample selection 
within the second level units is noninformative. It is 
important to emphasize that standard weighting of the 
sample measurements by the weights w,, = T,, which is 
routinely applied for single level models yields consistent 
estimators only for B. 
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2.3 State-space Models 


State-space models as considered here consist of two sets 
of equations: 


1. The measurement (observations) equation: 
yy —EGBy + Ld, a E,> E(,) = 0, 


Be gt Odd ats |, stl (2.3) 
2. The transition (system) equation: 
OC a.G a7. 1,52 Gy) =0, 
Egy ye ae Ort = 1a (2.4) 


where 6,=1 for k = O and 6, =0 otherwise. We also 
assume E(€,n,) = 0 for all ¢ and s. Note that both y, and a, 
can be multivariate. The measurement equations relate the 
observations y, at any given time point to covariate values 
X, with fixed (nonstochastic) vector coefficients B,, and 
linear functions L, of an unobservable state vector a,. The 
transition equations describe the time series relationships 
between the components of the state vector. The matrices 
X,, L, and G, are assumed to be nonstochastic although 
they may change over time, as is the case with the vector 
coefficients B,. Notice that the latter vectors can be 
included as part of the state vectors by taking their 
transition matrix to be the zero matrix of corresponding 
order and defining the corresponding residual variances in 
Q, to be very large. See Sallas and Harville (1981) for 
details. 

Although not written here in its most general form, the 
state-space model defined by (2.3) and (2.4) is known to 
include as special cases many of the time series and mixed 
linear models in common use. As important examples we 
mention the family of ARIMA models and models with 
random regression coefficients. The MLM defined by (2.1) 
can also be easily structured in a state-space form. To see 
this, replace the index i by f and define L, = [X,, Z,], 
a, =[B/,u/]', H, =0°Z,,and G, =[J,,0,] where J, and 0, 
define the identity matrix and the zero matrix of the 
appropriate orders. (The matrices Z, and X, are defined 
below (2.2).) The vector coefficient B, is added for conve- 
nience to the state vector. The covariance matrix Q, is 
block diagonal with 0, and Z,QZ/ as the two blocks. The 
use of the zeroes matrix 0, for the covariance of (B, - B,_,) 
guarantees that the B-coefficients are fixed over time, in 
accordance with (2.1). (The representation of the MLM in 
a State-space form is not unique.) 

For given covariance matrices {H,, Q,} and assuming 
that Bak and G, are known for all ¢, the best linear 
unbiased predictor (BLUP) of the state vector at any given 
time ¢, based on all the data accumulated until that time, is 
conveniently obtained by means of the Kalman Filter. Let 
G@,_, define the BLUP of a,_, based on the observations 
until time (f-1), with covariance matrix P,_,= 


57 


Cov(é,_, - a.,_,). The BLUP of a, at time (¢-1) is then, 
fy = 6 i, with covariance matrix Lar tes eg 
Cov(@,,,_,-a,) = G,P,_,G/+Q,. When new observations y, 
become available, the predictor G.,,-; and the correspon- 
ding covariance matrix are updated as 


A A 


= pol A 
A= Oy) ih cep Pe) F, (y, me Ay Sle Oca) 


P.=P 


t tlt-1 


ay a AO ia a 


tlt-1"-t agers cot 


(2.5) 


where Fi, = LP ugh; i H, fe Var(y, Vine) with Vate-1 = 
X,B, + L,6,,,_, defining the BLUP of y, at time (t-1). The 
actual application of the Kalman filter requires a proper 
initialization for .,,, and P,,, which depends on the model 
under study. See section 4 for the initialization under the 
model proposed in this paper. 

The unknown model parameters (8,, elements of a, 
and possibly L, and G,) are ordinarily estimated by MLE 
with the likelihood conveniently constructed by use of the 
“prediction error decomposition”. Assuming _ that 
dim(y,) =n, the log-likelihood takes the general form, 


vb 
log (L) = ~ (72 log (2n) sd log IF | 


t=1 


1 sues . 
SE Va Fr = Yq )}- 2-8) 


For a thorough discussion of state-space models and their 
applications, see Harvey (1989). 


3. A MODEL FOR HIERARCHICAL 
LONGITUDINAL DATA 5 


In this section we propose a time series multilevel model 
which combines separate cross-sectional two-level models 
by modelling the evolution of the first and second level 
random effects over time. Let S, define the sample 
available at time ¢, composed of m, level 2 units with n, 
level 1 units in level 2 unit h. The formulation of the overall 
sample in terms of the subsets S, covers situations where 
the longitudinal observations are collected at different time 
periods. The proposed model allows also for the rotation 
patterns mentioned previously and for wave non-response. 
Note that the samples observed at different time points are 
generally not disjoint and that the assumption that n, is 
fixed over time is not restrictive. Pfeffermann and Nathan 
(forthcoming) consider the case of temporal missing data 
for which this supposition does not hold. As long as the 
missing data are missing completely at random, generali- 
zation of the present methodology to this case is straight- 
forward. We assume the following two-level model to hold 
for the sample S,: 


/ / Ud 
a + 
Y nyt XY : Zi) Zn © nyt 


(teats Pg Pe A pec) be (3.1) 
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where y,,, is the outcome for first level unit 7 in second 
level unit ‘h, Xi and z,, are fixed known covariate vectors 
of dimensions p and q respectively, y, and v, are fixed 
(unknown) vector coefficients and u,, and e,., are inde- 
pendent second level and first level random effects. For 
given time t, The model defined by (3.1) is basically the 
same as the MLM model defined by (2.1), except that we 
ASSUME 2), = 2), for all j and ¢, thus distinguishing between 
first level covariates and second level covariates. We 
assume also for convenience Zp, = 1. The model is quite 
general in that all the covariate variables, the fixed vector 
coefficients and the random effects are allowed to vary over 
time in ways defined below. Notice that by assuming that 
(3.1) holds for the sample data, it is implicitly assumed that 
the sampling design is noninformative. See the discussion 
in section 2.2 and also section 4 below. 

As in (2.2), the model defined by (3.1) can be formulated 
in matrix form as, 


Y, =n, “An +L May +1, ene (3.2) 
where Yin =Drie- i Yin, t yx, = [Xp ioe Xinel > Z, =1ez,, 
and’ €, = [epg sic Cnn, Ait wath ® defining the Rronecver 


product. The matrix representation (3.2) can be written 
concisely as, 


Yipes pee Z 0 


(3.3) 


ht? 
where 2, Ge 
[Upp ej} ; 
Next we model the time series relationships of the vector 
coefficients and the random effects. We assume that the 
vectors B,,¢ = 1,2... are fixed without specifying the way 
they evolve over time. This assumption is generally not 
restrictive because in practical applications the overall 
sample size in any given time point is usually sufficiently 
large to allow accurate estimation of the vector coefficients 
without having to borrow information across time. For the 
random second and first level effects we postulate first 
order autoregressive [AR(1)] relationships of the form, 


= [Xi Zar] A al Fred ised by Piri be 


Unt = Au, 5 Si Cay Pen 1-1 i En (3.4) 
where A is a (qx q) matrix of fixed coefficients, pisa 
fixed scalar and 6,, ~NO,, A), 2, = NO! o i ol are 
independent white ares series. The fod denned’ by (3.4) 
is rather simple and as a further simplification we assume 
that A and A are diagonal, implying that the second level 
random effects are independent. It is assumed also that 
Ipl<1 and 1A,,!<1 for all & to guarantee stationarity. 
More complex models can be considered in principle but it 
should be emphasized that unlike in classical (aggregate) 
time series analysis, longitudinal observations may only be 
taken over a very short time period in which case the use of 
models that incorporate lagged values of high order may no 
longer be operational. For example, in the quarterly Israel 


LFS described in the introduction, individuals are in the 
sample for a total of 4 quarters over a time period of 6 
quarters which clearly limits the class of time series models 
that can be postulated for the random effects. 

The AR(1) models defined by (3.4) can be written 
concisely as 


Q,, = GO, 94 + props doesnt (3.5) 
where 
Ay 0 Sit 
G wi ’ 1] = ? 
h O npr ht bi | 
A 0 
ONO, Q,), QO, = 2 : 
eet s.6) 


By writing the proposed model using the equations (3.3), 
(3.5) and (3.6) and setting Z,,=L,,, H,,=0, it is easily 
seen to belong to the class of states space models presented 
in section 2.3, with no residual errors in the measurement 
equation. The model is defined for distinct second level 
units / but unlike in classical time series analysis where the 
data consist of a single long series, the data in our case 
consist of many independent short (longitudinal) series that 
could be observed over different time periods. Note that the 
transition matrix, G, and the covariance matrix, Q,, depend 
on h/ through the second level size n, but they are time 
invariant. In situations where the second level sizes are not 
fixed over time (for example, because of missing data), 
these matrices also change accordingly. 


4. ESTIMATION OF THE MODEL 
PARAMETERS 


In principle, the likelihood function holding for the 
model defined by (3.3), (3.5) and (3.6) can be maximized to 
obtain the maximum likelihood estimators (MLE) of all the 
unknown model parameters. However, the number of esti- 
mated parameters would usually be very large, which can 
intensify the computations and result in statistically un- 
stable estimators. For instance, even for p=q=2 and 
T=10 there are already 46 unknown parameters. We 
propose therefore a two-stage estimation procedure that 
employs MLM estimation for the “cross-sectional para- 
meters” and state-space model estimation for the “time 
series parameters”. The use of this procedure has the further 
advantage of accommodating appropriate weighting to 
protect against informative sampling. 

The procedure starts off by fitting the MLM defined by 
(3.1) to each sample S, separately, to obtain IGLS estimates 
of the time-dependent fixed effects B, = [y,,v/]’ and the 
variances of the random effects u,, and ent: Notice that by 
(3.4), 
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Var(u,,) = A* =(1- A?) 1A; 


Vat (€4,) = os lees p*)o, 


(4.1) 
using familiar relationships holding for AR(1) models. The 
use of this step yields estimates {B,, A7,6 6} for 
{B,, A", o. ey} ge ke Under the model, the true 
variances (A‘,o ) are fixed over time and assuming that 
the sample sizes at the various time points are fairly 
constant, the estimates A; and & can be averaged to yield 
single estimates 


(4.2) 


In the second stage the remaining parameters are 
estimated by maximizing the likelihood of the combined 
model defined by (3.3) (3.5) and (3.6), with the parameters 
estimated in the first stage held fixed at their estimated 
values. Since observations on different second level units 
are independent, the log-likelihood has the form 
log(L) = Y, log (Z,) where L,, the contribution to the 
likelihood from second level unit h, is defined by (2.6) with 
the index h added to all the components thus distinguishing 
between different second level units. As pointed out before, 
the number of time points for which the second level units 
are observed and the time periods over which the obser- 
vations are taken may differ between units so that the 
notation T in (2.6) for the number of time points needs also 
to be changed to 7,. 

When fitting the model to data obtained from rotating 
panel sampling designs as in the empirical study of the 
present paper, a further modification is required to account 
for the intermediate periods without observations. For 
example, for the Israel LFS described in the introduction, 
with rotation pattern of two quarters in the sample, two 
quarters out of the sample and two quarters in again, 7, = 4 
but the transition equations from ¢=2 to ¢=3(the next 
quarter with observations) have to be changed to account 
for the two quarters with missing observations. Repeated 
substitutions in (3.5) yield the following relationships: 


3 * * * 
O,3=G_O,o+Nn3> Na3~NO, O35) 


meriect Ae DA 0 
O13 = 0 (p* + p?+1)oe1, : 


(4.3) 


In order to apply the Kalman filter and compute the 
likelihood, it is needed to set initial values for a,,, and 
Po. This is simple under the present model as 
a, =[u,,,@),]’ is stationary with zero mean and covariance 
matrix defined by (4.1). Thus, the filter is started by setting, 
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iN OF Li rad 
= diag {J-A?)'A,o, (1 -p?)"'Y, ). (4.4) 


In the empirical study described in the next two sections we 
compare two methods regarding the set of parameters 
estimated in the second stage. 


Method 1: The parameters estimated in Stage 2 are the 
three AR coefficients p,A,,,4,, and the corresponding 
residual variances O. = = Var , ,) and A = Var(6,,), (equa- 
tion 3.6, three variances in total). Note that under this 
method the only estimates utilized from Stage 1 are the 
fixed parameter estimates {B, = 3 [¥,,0/]’}. By (4.1), the 
variances A* = Var(u,,) and o, = Var (é,,,) are estimated 
as 


(4.5) 


Method 2: The only parameters estimated in Stage 2 are the 
AR coefficients P41,,4,) (Equation 3.4). Note that with 
this method the variances A and om ag, set, in the likelihood 
as, A = (I - Ae De and o. Sate p)G utilizing (4.1), 
where S, and A* are defined by (4. 5), 

The estimation procedures described so far assume 
implicitly noninformative sampling. As discussed in the 
introduction, complex sample surveys often involve selec- 
tion with unequal probabilities that could be correlated with 
the values of the response variable. When this is the case, 
the model holding for the sample data may differ from the 
model holding in the population. A further advantage of the 
proposed two-stage estimation method is that it can be 
adapted to protect against informative sampling. This is 
done by applying the weighting procedure described in 
section 2.2 in the first stage, replacing the iterative IGLS 
algorithm by the PWIGLS procedure. Thus, for each 
sample S,, PWIGLS is used for estimating the MLM model 
parameters instead of using the IGLS. 


Comment 1: Informative selection of the first and second 
level units does not affect the conditional distributions of 
the random effects as defined by (3.4). Thus, although the 
distribution of u,, and e,, could be largely distorted 
because of the sample selection at time ¢ = 1, this has no 
effect on the distributions of u,,1u,,, or €,,1é,,. The 
implication of this property is that the computation of the 
likelihood in the second stage remains the same, but care 
should be taken of a proper initialization of the Kalman 
filter. As defined by (4.4), the filter is initialized by the 
unconditional means and variances of the random effects 
under the model, but at time ¢ = 1 the moments holding for 
units in the sample can be different because of the sampling 
effects. As is well known, for long enough series and under 
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some regularity conditions, the estimates derived from 
maximization of the likelihood are not sensitive to the 
initialization procedure but with short series, improper ini- 
tialization under informative sampling could distort the 
estimation process. Nonetheless, as illustrated in section 5, 
having a moderate number of longitudinal observations 
even of very short length (at most 4 observations in our 
application) and weighting the likelihood contributions by 
the inverse of the sample inclusion probabilities (applica- 
tion of the pseudo likelihood approach) yields approxi- 
mately unbiased estimators for all the time series model 
parameters. 


5. SIMULATION RESULTS 


In this section we report the results of a Monte Carlo 
study carried out for assessing the performance of the 
various estimation procedures described in section 4 under 
noninformative and informative rotating sampling schemes. 


5.1 Description of Simulation Study 


A) Generation of population data and sample rotation 
scheme 


Population values have been generated for individuals 
(first level units) within households (second level 
units), using the model defined by (3.1) and (3.4) (see 
below). The number of persons 1, observed within 
household h was selected at random with possible 
values of 2, 3 or 4. A new panel of households has 
been generated in each of 11 quarters and a sample of 
these households has been observed following the 
Israel Labor Force Survey rotation scheme of two 
quarters in the sample, two quarters out of the sample 
and two quarters in again. As easily checked, this 
process yields a complete sample of four panels in 
each of the quarters 6-11, with one panel in each 
quarter observed for the first time, one panel observed 
for the second time, one for the third time and one for 
the fourth and last time. (In the first quarter there is 
only one panel, in the next three quarters there are two 
panels and in the fifth quarter there are 3 panels.) In 
what follows we only consider the data observed for 
quarters 6-11. 


B) Population model 


The model used for generating the y-values for a 
given household h is defined by (3.1) and (3.4) with 
Xn = (Xy/1 »Xj9) and z,,=(3,z,,), such that the 
covariate values are fixed over time. The x-values 
were generated independently from the uniform distri- 
bution U[1, 2]. Values Z,. were generated from the 
uniform distribution U[1, 5]. In order to simplify the 
presentation and evaluation of the results, we also set 
the model coefficients to be time invariant such that 
¥,=Y =(6, -2)' and v, = v = (1, 2)’. The random error 


terms were generated independently between house- 
holds using the model (3.4) with A = diag[0.5, 0.7], 
A =diag[0.8, 0.5], p=0.4 and o, =0.25. Notice from 
(4.1) that Var(u,,)=A* =diag[1.067, 0.980] and 
Var (Cy) = Oo. = 0.298. 


C) Sample selection 


We consider two separate sampling schemes. 
Cl) Noninformative sampling: 


Population values have been generated for panels of 
30 households, with all the households belonging to a 
given panel selected to the sample and observed 
following the sample rotation scheme described in A 
above. The total number of sampled households in 
each of the quarters 6-11 is therefore m = 120. All the 
individuals belonging to a given household have been 
observed, yielding an expected sample size of n = 360 
individuals for each of the quarters. This sampling 
scheme corresponds to simple random sampling of 
households and individuals within the selected 
households. 


C2) Informative sampling 


Population values have been generated for panels of 
55 households. Households with random effects 
Up, <O (the value of the first random effect at the 
first time point) have been sampled with probability 1, 
households with random effects u,, , >0 have been 
sampled independently (Poisson sampling) with 
probability 0.1. All the individuals belonging to a 
sampled household have been observed. This 
sampling scheme yields an expected sample size of 
approvimately 30 households per panel and expected 
sample sizes of approximately m = 120 households 
and n = 360 individuals per quarter, similarly to the 
sampling scheme Cl. 


Comment 2: It should be emphasized that even though there 
are 4 panels observed in each of the quarters 6-11, there are 
only 11 separate panels that are used for estimation of the 
model parameters. Moreover, out of the 11 panels, only the 
panel entering the sample in quarter 6 for the first time is 
observed in 4 quarters, only 2 panels are observed in 3 
quarters, 6 panels are observed in 2 quarters and 2 panels are 
observed in only one quarter. This implies a total of 13 panel 
transitions, with about 390 household transitions observed 
for estimation of the time series parameters. (By a panel 
transition we mean that the same panel is observed on two 
occasions. For 3 of these panel transitions there is a time gap 
of 2 quarters between the two observations). We refer to this 
sample structure when assessing the estimation of the time 
series model parameters. 

The whole process of generating population values and 
selecting the sample has been repeated 100 times for each 
of the two sampling schemes C1 and C2, with one sample 
selected from each population. For each sample we applied 
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the two estimation procedures described in section 4. The 
simulations were run using the Gauss software package. 
Maximization of the likelihood has been carried out using 
the numerical optimization procedure, OPTMUM. 


5.2 Results 


The results of the simulation study are summarized in 
Tables 1-4 as averages over the 100 samples selected under 
the two sampling schemes. Each table contains the mean 
estimates of the model parameters, the empirical standard 
deviations (SD) of the estimators and the conventional 
t-statistics obtained by dividing the difference between the 
mean estimates and the true parameter values by the 
standard errors (SE), computed as SD/10. Notice that the 
estimates of the fixed vector coefficients B, = (y/,v/)’ are 
the same under the two estimation methods. 

Perhaps the most important outcome of this study, 
revealed from Table 1, is that under noninformative 
sampling it is indeed possible to fit successfully simple but 
nontrivial time series models to very short longitudinal 
series, provided that the number of observed series is 
sufficiently large. (The model is not trivial because even 
after subtracting the fixed effects, the dependent response 
variable is the sum of three AR(1) processes.) This 
conclusion is further strengthened by the fact that 8 out of the 
11 panels have been observed for at most 2 times, yielding a 
total of 13 panel transitions, three of which with a gap of 2 
quarters. See Comment 2 at the end of section 5.1. 


Table 1 
Means, Standard Deviations (SD) and ¢-Statistics of Estimators 
Under Two Estimation Methods. Noninformative Sampling 


Method 1 Method 2 
True eee we 
Parameter Mean SD t-statistic Mean SD _ t-statistic 
Value 
Vs 6.000 6.002 0.03 0.677 6.002 0.03 0.677 
> -2.000 -2.000 0.03 0.078 -2.000 0.03 0.078 


1.000 0.989 0.08 -1.357 0.989 0.08 -1.357 
V, 2.000 2.008 0.08 0.997 2.008 0.08 0.997 


Ay 0.500 0.497 0.07 -0.391 0.491 0.07 -1.271 
A, 0.700 0.696 0.07 -0.532 0.695 0.07  -0.820 
Ais 1.067 1.054 0.08 -1.668 1.045 0.08 -2.677 


[bss 0.980 0.991 0.10 1.042 0.990 0.11 0.906 
p 0.400 0.398 0.02 -0.937 0.397 0.02  -1.637 
o 0.298 0.298 0.01 -0.062 0.297 0.01 -1.382 


Evaluation of the performance of the two sets of 
estimators in Table 1 shows that all the estimators under 
Method 1 are highly insignificant based on the conventional 
t-statistics and only the estimator of Aj, is significant under 
Method 2. Note that even in that case the absolute relative 
bias is about 2% and considering that MLE of time series 
parameters are generally not strictly unbiased, such a small 
bias in one of 10 parameters is expected. Notice also that 
the standard errors of the mean estimators under the two 
methods are very similar, a result observed also in the other 
tables. 
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Next we consider the case of informative sampling. 
Table 2 shows the results obtained when ignoring the infor- 
mative sampling process, using the same estimation proce- 
dures as used for the noninformative case. As indicated 
very clearly by this table, some of the parameter estimates 
are highly significant, particularly the estimators of the 
parameters indexing the time series model of the random 
effects u,,, that define the sample selection probabilities. 
Thus, we find that the absolute relative bias in estimating vy 
is about 27%, and large absolute relative biases are alse 
observed for the estimators of A,, and Ain: (The model 
defined by (3.1) can be rewritten as Viaje =X Vit ZL Uppst Cai 
where w,, = u,,+v,, such that for v, = v as under the simu- 
lation model, v, = a (u,1,))- Note that the three biases are 
negative, RES is explained by the fact that the selection 
mechanism utilized for this study oversamples individuals 
with observations that contain negative random effects 
Uni In this case again, the two estimation methods 
perform very similarly. 


Table 2 
Means, Standard Deviations (SD) and ¢-Statistics of Estimators 
Under Two Estimation Methods. Informative Sampling, 
Unweighted Estimators 


Method 1 Method 2 
Parameter ore Mean SD _ t-statistic. Mean SD t-statistic 
Value 
Y 6.000 5.998 0.02 -0.768 5.998 0.02 -0.768 
i -2.000 -2.000 0.03 0.104 -2.000 0.03 0.104 


1.000 0.728 0.09 -34.385 0.728 0.09 -34.385 
2.000 2.005 0.09 0.564 2.005 0.09 0.564 
0.500 0.438 0.09 -6.742 0.434 0.09 - -7.453 
A,» 0.700 0.738 0.09 4.078 0.735 0.09 3.941 
Ai 1.067 0.995 0.09 -7.766 0.994 0.09  -7.883 
Ay) 0980) 18003" O10, 27352) 902987 F0r10 0.698 

p 0.400 0.407 0.02 3.184 0.405 0.02 2.218 

o 0.298 0.298 0.01 0.644 0.296 0.01 -1.800 


Table 3 shows the results obtained when using the 
PWIGLS algorithm for the estimation of the MLM para- 
meters (section 2.2) and weighting the time series likeli- 
hood r, contributions ele )=- u W27 nh’ os (27) 

+ 1/2)" log lF,,| + 1/2(Y, ht Forte )' ht 08 Yie- J 
by the household sampling weights w, = 1/ Pr (hi Es), using 
the same 100 samples as used for Table 2. Weighting the 
likelihood contributions by the inverse of the sample inclu- 
sion probabilities is an application of the pseudo likelihood 
approach that is often recommended for fitting single level 
models to cross-sectional data, see, e.g., Binder (1983), 
Skinner et al. (1989) and Pfeffermann (1993). As revealed 
from this table, the use of the PWIGLS algorithm and 
weighting the likelihood eliminates the large biases 
observed in Table 2, despite the improper initialization of 
the Kalman filter with very short series. (See the discussion 
in Comment 1 at the end of section 4.) Here again, the two 
estimation methods perform quite similarly, yielding one 
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biased estimator in each case but with both biases being 
relatively very small. 

It is important to mention that the SD’s of the weighted 
estimators shown in Table 3 are always larger than the 
corresponding SD’s of the unweighted estimators displayed 
in Table 2. As pointed out by one of the referees, this 
implies that the empirical root mean square errors 
(RMSE’s) of the unweighted estimators in Table 2 are in 
fact larger than the empirical RMSE’s of the corresponding 
estimators in Table 3. This outcome, however, is due to the 
relatively small sample sizes employed in this study. For 
larger samples (larger numbers of households and indivi- 
duals within the households) the RMSE is dominated by the 
bias which, unlike the variance, is not reduced as the 
sample size increases. Thus, it is clear that as the sample 
size increases the RMSE’s of the weighted estimators 
become smaller than the RMSE’s of the unweighted esti- 
mators. The fact that probability weighted estimators have 
larger variances than the corresponding unweighted estima- 
tors is well known from many other studies, see 
Pfeffermann (1993) for discussion and references. 


Table 3 
Means, Standard Deviations (SD) and f-Statistics of Estimators 
Under Two Estimation Methods. Informative Sampling, 
Weighted Estimators 


Method 1 Method 2 
Parameter pie Mean SD _ t-statistic Mean SD __ frstatistic 
Value 
vi 6.000 5.997 0.04 -0.607 5.997 0.04 -0.607 
¥> -2.000 -2.000 0.05 -0.007 -2.000 0.05 -0.007 
v, 1.000 0.978 0.14 -1.518 0.978 0.14 -1.518 
Vv, 2.000 2.019 0.14 1330" 823019) 40.44 1.330 
ay 0.500 0.490 0.15 -0.695 0.477 0.14 -1.611 
AD 0.700 0.699 0.17 -0.066 0.709 0.16 0.545 
Ai 1.067 1.055 0.17 -0.664 1.040 0.17 -1.560 


Aj}, 0.980 1.023 0.19 
0 0.400 0.401 0.04 
o 0.298 0.297 0.01 


2199 SLOVO FF O19 1.571 
0.135 0.397 0.04  -0.813 
-0.486 0.294 0.01 -3.340 


As discussed in Comment 1 at the end of section 4, 
informative sampling distorts the cross-sectional distribu- 
tion of the sample observations and the initialization of the 
Kalman filter, but does not affect the conditional distribu- 
tions of the first and second level random effects defined by 
(3.4). Thus, it is interesting to test whether the use of the 
PWIGLS algorithm for estimating the cross-sectional model 
parameters but without weighting the time series likelihood 
likewise controls the bias. Table 4 shows the results 
obtained for this case with the same samples as used for 
Tables 2 and 3. The estimators of the fixed vector coeffi- 
cients B, = (y,,v,)’ are the same as in Table 3 and hence 
are not shown again. Notice that the estimators of A;,, A5, 
and o. under Method 2 are also the same as the comespon” 
ding estimators in Table 3. 

The interesting result revealed from Table 4 is that the 
estimators of A,, and 4,, have now a non-negligible bias, 


unlike the corresponding estimators in Table 3. This result 
can be explained as follows. Under the informative 
sampling scheme, the expectation of the random effects 

u,, , Corresponding to households / in the sample is below 
zero, E (Uy, , |hes) <0, and hence the initialization of the 
Kalman filter by the population expectation (Eu,, , =9, 
Equation 4.4) yields biased estimators. On the other hand, 
by weighting the likelihood contributions L, by the inverse 
of the sample selection probabilities, the proportions of 
likelihoods L, corresponding to random effects that are 
below and above the model expectation is balanced to the 
population proportions and thus the use of the model 
expectation for the initialization process does not bias the 
estimation process. As noticed for the previous tables, the 
SD’s of the unweighted estimators in Table 4 are much 
smaller than the SD’s of the corresponding weighted 
estimators in Table 3. 


Table 4 
Means, Standard Deviations (SD) and ¢-Statistics of Estimators 


Under Two Estimation Methods. Informative Sampling, 
Weighted MLM, Unweighted Likelihood 


Method 1 Method 2 
Parameter ime Mean SD t-statistic Mean SD __ tstatistic 
Value 
Ay 0.500 0.468 0.09 -3.477 0.453 0.10 -4.569 
IN 0.700 0.742 0.11 33948 O37. 01d 3.197 
AR 1.067 1.060 0.11 -0.598 1.040 0.17 -1.560 
MS 0.980 1.008 0.11 2.449 1.010 0.19 L571 
p 0.400 0.407 0.02 3.021 0.402 0.02 0.894 
o 0.298 0.298 0.01' 1.013 0.294 0.01 -3.340 


6. APPLICATION OF THE MODEL 
TOLFS DATA 


We fitted the model defined by (3.1) and (3.4) to an 
empirical data set extracted from data collected by the Israel 
LFS for Jerusalem during the years 1990-1994. The data 
contain complete records for 567 individuals in 475 
households, with each individual observed in four quarters 
according to the rotation pattern described before and used 
for the simulation study. Out of the 475 households, 385 
have one individual record, 88 have 2 individual records 
and only 2 households have 3 individual records. The out- 
come variable is y = number of hours worked during the 
week preceding the interview, (y = 39.8, sd (y) = 14.8; 
calculated over all individuals and all the quarters). The 
individual level auxiliary variables are x, = years of 
education, (x, = 13.4, sd (x,) = 4.8) and x, = gender, (41% 
females). The household level auxiliary variables are 
z, = 1 and z, = number of employed persons in the house- 
hold (Z, = 1.48, sd(z,) = 0.56). 

We estimated the model parameters using the two 
methods described in section 4. The sampling weights 
attached to these data are very similar across households 
and individuals so that we only computed the unweighted 
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estimators. The LGLS algorithm produced negative 
variance estimates for A;, in some of the quarters and these 
estimates have been set to zero when averaging the variance 
estimates under Method 2. The quarterly estimates of the 
fixed model coefficients have not been averaged as they 
change significantly over the five years period. 

The estimates computed by the two methods for the 
variances and autoregression coefficients are shown in 
Table 5 using the same notation as in the previous tables. 
The two sets of estimates are not very far except for the 
estimator of A;, which, has already mentioned was found 
to be negative in some of the separate IGLS runs. Note in 
this respect that for most of the households there is only a 
single individual record (see above), and that for almost all 
of these households z, = 1. This complicates the estimation 
process since for such households it is impossible to 
distinguish the first (individual) level effect from the two 
household effects, which are likewise confounded. (Note 
that the sum of the latter two variances is similar under the 
two methods.) As discussed below, the estimators in Table 
5 are dominated by the observations obtained for 
households with two individual records. 


Table 5 
Estimates of Variances and Autoregression 
Coefficients Under Two Estimation Methods. 


LFS Data 
Parameter _ Aj, AS Ai IN p Oo, 
Method1 0.915 -0.606 73.88 2.541 0.242 102.306 
Method2 0.976- -0.548 56.88 14.753 0.448 101.001 


Under the Israel LFS sampling design, each individual 
record consists of 4 observations taken in quarters 1,2, 5 
and 6, with quarter 1 defining the first calendar quarter f¢ 
that the individual is in the sample. In order to assess the 
prediction power of the model, we computed for every 
individual record (h,7) the empirical est when 
predicting the adjusted values 7, = ( Vn — Xhig a Z1g%q) 
using the household data observed for the doe alid 3 
quarters that the individual has been in the sample. Note 
that by subtracting the fixed effects from the original 
observations, the distribution of the adjusted values no 
longer depends on the calendar quarters. The innovation for 
quarter q is the corresponding prediction error which, by 
Bs ascomputed.as:d sa (rps 2 hips Te pliihig) = 

~ Zag : 1) O54) ee: 245; is eet , is the 
ee of the state vector a, =(u,_, a? hig y none the data 
eS until quarter g-m, with m= q- "1 for q = 2,6 and 

= 3 for q = 5. The predictor 4,,,_,, 18 obtained by 
inares of the Kalman filter with th the corresponding 
estimated parameters (see section 2.3 and Equations 3.5 and 
4,3). 

Table 6 shows the roots of the means of the square 
innovations (RMSI) by quarter and the number of house- 
hold (HH) records, as obtained under the two estimation 
methods (using the parameter values displayed in Table 5). 
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For comparison, we also show the RMSI’s of the innova- 
tions obtained by predicting the adjusted value for quarter 
q by the adjusted value in the preceding quarter. The 
“naive” predictor 7, =r, q-m can be interpreted as being 
the optimal predictor under the simple random walk model 
ria = ’h, qg-m + rror.The means of the innovations 
3 m) for g = (2, 5, 6) are (0.68, 0.24, 0.301) for 
novencise with one record, (1.24, -1.20, 0.60) for 
households with two records and (4.02, -5.82, 7.68) for 
households with 3 records but recall that the latter means 
are based on only 2 households. The corresponding means 
of the empirical innovations computed under the model are 
smaller in absolute value in all the cases. 


Table 6 
Root Mean Square of Innovations by Number of Household 
Records and Quarter Under Two Estimation Methods 
and Naive Prediction. LFS Data 


onl: ; : é 
Quarter 2 5 6 2 5 6 2 5 6 
Method 1 11.54 11.16 11.62 12.26 11.71 10.88 9.61 9.98 8.94 
Method 2 11.71 11.16 11.49 12.10 11.48 10.91 9,30 9.78 7.90 


Naive Pred. 14.00 11.92 13.60 14.71 15.12 13.47. 7.50 13.32 11.29 


The data analyzed in this section behave much more 
erratically than the data used for the simulation study 
generated under the model and we cannot claim that the 
model employed yields the best possible fit (see also 
below). Nonetheless, the values displayed in Table 6 
illustrate some important features of the model. We mention 
first the generally much better performance of the model 
predictors compared to the naive predictor Pig r, 
with the two estimation methods yielding similar 1 Ma. 
The superiority of the model is explained by the fact that 
whereas the first order autocorrelations of the two random 
household effects used for the model predictions are high in 
absolute value (very high for the first component), the 
autocorrelations of the adjusted values (the “total” errors) 
are only of moderate size. The first order autocorrelations 
of the random components are the corresponding 
autoregression coefficients, see Table 5. The empirical 
autocorrelations of the adjusted values, Cort (F 4; 7” Py, fe cay 
g = 2,5, 6; m= 1 for g = 2,6; m = 3 for q =5 are corres- 
pondingly (0.46, 0.59, 0.51) for one record households, 
(0.48, 0.36, 0.45) for two record households and (0.92, 
0.43, 0.63) for three record households (based on 6 
individual records). 

As already noted, the fact that most households have 
only one individual record introduces identifiability 
problems since for such households it is impossible to 
distinguish between the three random effects. Computation 
of the correlations Corr (7,4 A ny ), under the model 
using the parameter estimates in Table 5 shows a good fit to 
the correlations computed for two record households. This 
in turn illustrates that the estimators in Table 5 are 
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dominated by these observations and we conclude that the 
model fits best the observations obtained for the households 
with two records. Note, however, that the RMSI’s obtained 
for the other household sizes are not higher than the 
RMSI’s computed for the two record households (see also 
below). It is important to mention in this regard that if the 
data had been aggregated over all the individuals observed 
in a given calendar quarter, it would have been impossible 
to account for the random household effects, resulting in 
inferior predictions of the individual observations. See the 
discussion in the introduction. (Modelling the aggregate 
data is rather complicated in this case since the sample in 
each calendar quarter consists of 4 different panels as 
defined by the number of times that individuals are in the 
sample. This implies that the models holding for these 
panels are different, depending on the number of observa- 
tions available for each panel.) 

Other interesting results noted in Table 6 are that the 
RMSI’s under the model are generally lower for g = 6 than 
for g = 2, as explained by the use of more observed data for 
the same individual in the prediction process (more 
observed data for estimating the random effects in the 
preceding quarter). Also, for g = 6 the RMSI’s decrease as 
the number of household records increases, as explained by 
the use of data observed for other household members. 
Finally, the RMSI’s for households with 3 records are much 
lower by use of the model than the RMSI’s obtained for 
households with 1 and 2 records but we mention again that 
there are only 2 households with three records. The unex- 
pected results in Table 6 are that for households with one 
record the RMSI’s are somewhat larger for g = 6 than for 
q= 5 (note the relatively high and unexplained correlation 
of 0.59 between the adjusted values 3 quarters apart 
computed for these households), and that for g = 2 and q =5 
the RMSI’s for households with 2 records are larger than 
the corresponding RMSI’s for households with 1 record. 
With empirical data of relatively small size such anomalies 
are not unusual and they show up even more prominently 
with the naive predictor. (The fact that for a given number 
of household records the RMSI’s by use of the model for 
q = 5 are of similar magnitude to the other RMSI’s is 
reassuring given that the predictions in this case are 3 
quarters ahead.) 


7. CONCLUSIONS AND MODEL EXTENSIONS 


The results of this paper illustrate that it is possible to fit 
time series models to longitudinal series of very short length 
and with missing observations. The model used in the 
present study is an extension of the standard two level linear 
model by which both the first and second level random 
effects evolve stochastically over time. This kind of model 
is suitable for modelling longitudinal measurements that are 
taken for hierarchical populations. Application of the 
PWIGLS algorithm combined with standard probability 


weighting of the time series likelihood is shown to protect 
against the effects of informative sampling. 

Multilevel models are often fitted to discrete data, in 
which case the models contain nonlinear components. In 
principle, the two-stage estimation method proposed in this 
paper can be applied in this case as well, although with very 
short longitudinal series the range of models that can be 
fitted is obviously limited. Moreover, a common procedure 
for estimating the unknown model parameters in the 
discrete case consists of linearizing the nonlinear compo- 
nents on each iteration of the IGLS around estimates 
obtained on the previous iteration, and then applying the 
standard IGLS for computing the revised estimates. See 
Goldstein (1995) for details. Thus, it seems feasible to 
extend the PWIGLS algorithm to the discrete case without 
major difficulties. 

In this paper we have not considered variance estimation. 
This is no problem under the standard IGLS and 
Pfeffermann et al. (1998) propose simple variance 
estimators for the PWIGLS procedure. However, estimation 
of the variances of estimators obtained from maximization 
of the time series likelihood is more problematic because of 
two reasons. First, the possibly short length of the longitu- 
dinal series may no longer justify the use of the information 
matrix or permit stable estimation thereof, even with large 
number of second level units. Second, the MLM estimators 
are held fixed when maximizing the likelihood, implying 
that the MLE abstract from the sampling errors in the 
estimation of the MLM parameters. A possible solution to 
this problem is the use of re-sampling methods that allow to 
account for all sources of variation in the estimation 
process. 

Finally, we mention an important application of the 
proposed model for the imputation of missing data. In a 
recent article, Pfeffermann and Nathan (forthcoming) 
illustrate the large reductions in the imputation variance that 
can be achieved under the model compared to the use of 
more standard imputation methods that ignore the common 
household effects. 
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A Conditional Mean Squared Error of Small Area Estimators 


LOUIS-PAUL RIVEST and EVE BELMONTE! 


ABSTRACT 


This paper suggests estimating the conditional mean squared error of small area estimators to evaluate their accuracy. This 
mean squared error is conditional in the sense that it measures the variability with respect to the sampling design for a 
particular realization of the smoothing model underlying the small area estimators. An unbiased estimator for the conditional 
mean squared error is easily constructed using Stein’s Lemma for the expectation of normal random variables. This 
estimator can be calculated for any shrinking strategy; composite and empirical Bayes estimators are considered in this 
work. It can be calculated when the small area estimators are benchmarked to coincide with direct estimators at hi gh level 
of aggregation. It can accommodate skewness in the data and estimated variances. The conditional mean squared error 
estimator does not rely on any smoothing model. The price to pay for this property is a high variance; the new estimator 
is unstable under heavy shrinking. In these situations, it still provides useful diagnostic information about the shrinking 
model. It can also be seen as a building block for estimators of unconditional mean squared errors such as Prasad and Rao’s 
(1990). Examples dealing with the estimation of the under-coverage in the Canadian Census illustrate the application of 
this new estimator. 


KEY WORDS: Census under-coverage; Diagnostics; Empirical Bayes estimation; Estimated variances; Skewness; Stein’s 
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lemma; Survey sampling. 


1. INTRODUCTION 


In survey sampling, the need to develop accurate meth- 
ods of estimation for small areas poses challenging statis- 
tical problems. For small areas, direct survey estimates 
have too large a variance to be reliable. Small area tech- 
niques “improve” direct estimates by shrinking them to- 
wards model based smoothed values. Simple shrinking esti- 
mators are proposed by Purcell and Kish (1979). In a 
pioneering paper, Fay and Herriot (1979) demonstrate that 
this can lead to interesting gains in precision. The review 
papers of Ghosh and Rao (1994) and of Singh, Gambino 
and Mantel (1994) provide convincing evidence of the 
Vitality of this area. 

The estimation of the errors in small area estimation is 
receiving an increasing attention, see Singh, Stukel and 
Pfeffermann (1998) and Booth and Hobert (1998). This 
paper suggests estimating the conditional mean squared 
errors of small area estimators. The conditional mean 
squared error can be estimated for all shrinking strategies, 
either empirical Bayes or decision theoretic (Purcell and 
Kish 1979). Other mean squared errors, such as Prasad and 
Rao’s (1990), and Singh, Stukel and Pfeffermann (1998) 
frequentist proposals measure the variability with respect to 
both, the sampling design and the smoothing model. The 
mean squared error of this paper is conditional in the sense 
that it measures variability with respect to the sampling 
design for a particular realization of the smoothing model. 
This feature is attractive since the conditional estimator 
reflects the conditions under which the survey has been 
carried out (see Sarndal, Swensson, and Wretman 1992, ch. 


7). The drawback of this property is a high variability. In 
some instances, the proposed estimator is too variable for 
practical use. 

When shrinking is important, the conditional mean 
squared error estimators are highly unstable. An uncondi- 
tional assessment of the precision of small area estimators 
must be used. In this situation, the conditional estimator 
proposed in this paper still provides some useful informa- 
tion. It can be looked at as a diagnostic for comparing 
smoothing models. It can also be a building block for 
constructing Monte Carlo estimates of unconditional mean 
squared errors in situations where closed form formulas, 
such as Prasad and Rao’s (1990), are not available. 

The assessment of the accuracy of estimators for the 
under-coverage, at the provincial and sub-provincial levels, 
of the Canadian Census motivated this work. Alternatives 
to the direct estimates for provincial under-coverage are 
discussed by Royce (1992) and Rivest (1995). Dick (1995) 
applies empirical Bayes methods to sub-provincial under- 
coverage estimates. These two examples are treated in 
section 5. 

An estimator of the conditional mean squared error is 
presented in section 2. Its construction relies on the multi- 
variate version of Stein’s Lemma for the expectation of 
normal deviates. Section 3 suggests changes to the condi- 
tional estimator to accommodate skewness in the distribu- 
tion of the direct estimators and estimated variances. 
Section 4 discusses the application of the new estimator to 
empirical Bayes estimators. Its relationship with Prasad and 
Rao (1990) prediction variance is highlighted. Examples 
are treated in section 5. 


Louis-Paul Rivest and Eve Belmonte, Département de mathématiques et de statistique, Université de Laval, Ste-Foy, Québec, Canada, GIK 7P4. 
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2. A CONDITIONAL MEAN SQUARED 
ERROR ESTIMATOR 


Suppose that there are n small areas and let 
Lt =([,,.-..H,,)' denote the unknown population characteris- 
tics for these small areas. The direct survey estimates for 
the 1 small areas are y = (y,, ...,y,,)' where the distribution 
of yis N,(u,2), an-variate normal distribution with mean 
vector p and known variance-covariance matrix 2. As 
pointed out by Ghosh and Rao (1994), the normality 
assumption is likely to hold for many surveys since direct 
survey estimates are usually functions of sums of variables. 
The nxn matrix © is a design based measure of precision 
for y. For the time being, this matrix is assumed to be 
known. This assumption is relaxed in section 3.2. The 
uncertainty in y comes from the random selection of the 
sampling units. Subscript S, for sampling design, denotes 
expectations taken with respect to the distribution of y. 

In atypical application of small area techniques, one has, 


ee Lewis 
' Dy Wi 


where y i is the y-value for the j-th sample unit in small area 
i, Wi, is its sampling weight and the sum is over all the 
sample units in small area 7. In many instances, the variance 
covariance matrix & is diagonal; its (i,i) term, is 6,, = 
Var, (y,); when they are non null, the off diagonal elements 
of & are denoted by o,,,i,j = 1,..., 7. 

Several methods have been proposed to improve the 
accuracy of direct survey estimators. They involve 
shrinking y, towards some indirect estimator of p,. The 
resulting estimators can be written as 


preys iw Jibei 7 (1) 


where the g.’s are functions depending on the shrinking 
strategy. 

In vector form, one can write (1) as fi = y + g(y) where 
g, whose i-th component is equal to g,, is a function 
defined from R” to R”. We assume that for each 7, the 
right partial derivative and the left partial derivative of g, 
with respect to y. exists for any yin R”. When they are 
equal, dg,(y)/ oy, denotes the common value; if they differ 
dg, (y)/ oy, is the average between the two values. The 
component of g(y) and their partial derivatives are 
assumed to have finite variances. A conditional assessment 
of the precision of fi as an estimator for p is given by the 
matrix of the mean product errors which is given by 


Es{(h - p(B - w)'} =2+ £o{(y - pw)g(y)'} 


+Es{g(yyy -p)'} + Eolfg(yg0)} 
On the right hand side of this equality, the only quantities 


for which there are no obvious estimators are 
EA(y-p)g(y)'} and E,{g(y)(y-p)'}. Their evaluations 
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are eased by the following result which is a multivariate 
extension of Stein’s lemma (Stein 1981). Its proof is given 
in the appendix together with the proofs for Propositions 2, 
3, and 4. 


PROPOSITION 1: Let y be a VN (uy, 2) random vector 
then, 


Es{Q - wWg(y)'} = ZEs{Ve(y)}, 


where Vg(y) is an m Xn matrix whose (i,j )-th element is 
given by g/'(y) = og,(y)/dy,. 

Now according to Proposition 1, 2Vg(y) is an unbiased 
estimator for E.{(y-w)g(y)'}. Thus the conditional 
estimator (index “‘c” stands for conditional) for the matrix 
of the mean product errors is given by 


mpe,(fl) = 2 + LVg(y) + VgQy'Z+g(ygy)’. (2) 


The diagonal terms of (2) can be negative. Since they 
estimate mean squared errors, a better estimator for the 
mean squared error of {i, is 


mse; (fi,) = max 0,6, +20,{2'(y) +g/()} +g, 0) | 
J 


It generalizes an estimator proposed by Bilodeau and 
Srivastava (1988) for James-Stein estimator, and by Robert 
(1992 p. 292) for empirical Bayes estimators. When the 
y,’s are independent, with 6, = 0 when 7 #7, then 


og,(y) ‘ 


ee i 2 

mse, (H;) = 6, + 26, gi, (3) 
I 

and mse, (fi,) = max{mse,({i,), 0}. 

Kott’s (1989) small area estimator has g,(y) = a, (Y; - yi), 
where ¥, is a measure of location for the y’s and @, is a 
smoothing parameter. These two statistics involve variance 
estimates calculated at the “unit” level, that is using the 


Vij *s. Kott’s (1989) conditional mean squared error is 


v(fi,) = 6, - 26) a (4,(y, ape 


This is equal to (3) when both (d/dy,) 4, and (d/dy,)¥, are 
null. Thus Kott’s (1989) estimator for the conditional mean 
squared error does not account for the estimation for the 
variance components. This may account for the biases that 
it exhibited in the simulations reported by Prasad and Rao 
(1999). 

The estimates mse, and mpe, can be evaluated numeri- 
cally by taking 


ON 19 Vj * Ejay ey) 
og, (y) 2 Tey 9009 Vij re © Vjap eV) 
dy, 2€ 
where € is a small positive number. Thus mse, and mpe, 


can be calculated in all circumstances, even when g has no 
explicit form. 
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To illustrate the flexibility of the conditional estimator, 
consider fi" = fi(Zy,)/(2Gi,), an estimator bench-marked 
to agree with the direct estimator for the y-total. One has 
fi" =y + g"(y) where 


aes DA) LY, 
Witney eee ili 
2s H; yy HM; 
It might be difficult to derive an analytical formula for 
mpe,(il"), however this expression is easily evaluated 
using numerical derivatives. Modifications of the condi- 


tional estimator to account for non-normality in the y,’s and 
for estimated variances o., are given next. 


3. SENSITIVITY ANALYSIS 


In many surveys, especially those in the business sector, 
the study variables are skewed. Some of this skewness 
might still be left in the direct estimators y,. This section 
suggests a correction to the conditional mean squared error 
to account for skewness in the distribution of y. It also 
proposes ways to account for the estimation of the variances 
6,, in the mean squared error calculations. 

In practice the variances o,, are estimated. Several 
authors (Dick 1995; Hogan 1992) smooth the variances 
before calculating the small area estimates. They then 
consider the smoothed variances as the true variances in the 
small area calculations. Section 3.2 gives a condition under 
which replacing the estimated variances by their smoothed 
values yields unbiased mean squared error estimators. It 
also consider situations where the sampling variances are 
estimated with random groups (Wolter 1985 ch.2). This 
method consists in carrying a certain number, say k, of 
replications of the survey design. This yields, for each i, k 
estimates of 1; 6,, is then equal to the sampling variance 
of these k estimates divided by k. Assuming that these k 
estimates are normally distributed, one can consider that, 
suitably normalized, the distribution of 6,, is chi-squared 
with k - 1 degrees of freedom. A conditional mean squared 
error, adjusted for variances estimated with random groups, 
is proposed in this section. To keep the discussion simple, 
we assume in this section that © is a diagonal matrix; in 
other words the y,’s are assumed to be independent random 
variables. 


3.1 Non-Normality in the Distribution of y, 


In many applications of small area estimation, the distri- 
butions of the y,’s are not exactly normal. A simple adjust- 
ment to (3) is proposed to deal with asymmetry in the 
distribution of the y,’s. 

Suppose that the skewness of y,, p,=E5{Q; =p) }/ 
ae is small and non-zero. A first order Edgeworth series 
for the distribution of y, is given by (see for instance Reid 
1991): 


69 


cal GA EAC 
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er reamedsa 8 paint aes 

f Gi; Gi; 
Such an expansion is used to correct for skewness in the 
direct estimators (Barndorff-Nielsen and Cox 1989, remark 
2 p. 92). Expansions involving additional terms are used 
for correcting for both skewness and kurtosis; they will not 
be considered in this section. The evaluation of 
E{(y, -u,)g,(v)} under f needed for the construction of 


the conditional mean squared error estimator, is given in 
Proposition 2. 


PROPOSITION 2: When y, distributed according to f(t), 
as p, tends to 0. 


E{(y, - 4)g;0)} = 


a ; Gu : 
op 207) Cie Bs E 
dy, 2 


Peay 


o’g,(y) 
dy, 


A mean squared error estimator corrected for asymmetry 
is therefore given by mse, (fi,) = max{0, mse, (fi,)} where 


+ O(p;). 


sn 28;(Y) 

i I 2 i 
Yi oy, 

In practice, it might be difficult to find individual skewness 

coefficients p, for each i. A better strategy might be to 

combine all the data points to come up with a common 

p-value. 


g (vy. 


mse, (H1,) = 6, + 26, 


og,(y) ‘ 
a 


3.2 Estimated Variances 


Consider first a survey where the 6,,’s are estimated 
using k random groups. Assuming normality, one can 
consider that {(k - 1)6,,/06,,:i=1,...,m} is a sequence of 
independent y7_, random variables which is independent of 
y. Evaluating the conditional mean squared error (3) with 
variance estimates 6,, yields potentially biased estimators, 
since g,(y) and its derivatives depend on 6,,. The potential 
bias can be expressed as 


og. og. 
2 ra 2,84 Saat fi 
i oy, oy, 


As shown in the Appendix, this bias is O(1/k). The next 
proposition suggests a small change to (3) that reduces its 
bias (4). 

PROPOSITION 3: Replacing 6,, by (k - 1)6,,/( + 1) in 
the evaluation of 0g,(y)/dy, for calculating the mean 
squared error estimator (3) yields an estimator with an 
O(1/k*) bias. 
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The correction factor (k - 1)/(& + 1) has been proposed in 
a different context by Scott and Smith (1971). Other 
methods are available for correcting the bias for estimating 
variances, depending on the way in which G,, is estimated. 
For instance if the 6,, are independent N {o,, var (G,,)} 
random variables distributed independently of y, then by 
Stein’s lemma, (4) is equal to 2var (6,,)E{0*g,(y)/dy,06,}. 
Suppose now that the variances are estimated, not 
necessarily with random groups. In surveys, such as those 
considered in Dick (1995) and Hogan (1992), explanatory 
variables are available to model estimated variances. Small 
area estimators are then calculated with the predicted 
variances G,, under the smoothing model; this means that G,, 
enters in the calculation of g, in (1). Considering (4), the 
mean squared error estimated with the smoothed variance, 


dg. 
ahscin Ig (y) 


i ii 


+ g(y) 


is unbiased provided that 


ee 
26, -6,;) a }-0 


i 


When g,(y) is calculated with smoothed variances, (4) 
should be small; the above condition holds provided that 


og, 
ae 5;;) z 2 <a 0, (5) 


where index V refers to the model for smoothing the 
variances. One can easily test whether this condition holds 
by calculating the correlation between the variance 
residuals and the partial derivatives of the functions g,. 
Since, as shown in Proposition 5 of the next section, 
unconditional mean squared errors can be derived as 
expectations of mse,(fi,) testing whether (5) is true is 
relevant even when unconditional measures of accuracy, 
such as Prasad and Rao’s, are calculated. Indeed, replacing 
variances by their predicted values biases the mean squared 
error estimators, conditional or unconditional, when (5) is 
violated. 


4. MEAN SQUARED ERROR ESTIMATION 
FOR EMPIRICAL BAYES ESTIMATORS 


4.1 Model Construction 


This section assumes that the y,’s are independent, i.e, 
that X is diagonal. In an empirical Bayes setting, the model 
(M) for smoothing direct estimators expresses the para- 
meters p1,’s as random variables whose distributions depend 
on a p-variate auxiliary variable x, (Maritz and Lwin 1989, 
chapter 3), 


UW, =x;B + v,, (6) 
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where B is a p x 1 vector of unknown regression parameters 
and the v,’s are independent random variables with mean 0 
and variance o. Often the v,’s are assumed to be normally 
distributed; the marginal distribution of y,, with respect to 
both the sampling design S and the smoothing model M, is 
then NV (x! bio, +0 o) The empirical Bayes estimators are 
obtained by shrinking the direct estimators y, towards their 
predicted values under (6). 

The extent of the shrinking depends on estimators of the 
parameters of (6) calculated from the marginal distribution 
of y,. Several methods are available for parameter estima- 
tion (Cressie 1992). A popular estimator for o (see Lahiri 
and Rao (1995)) is 


0, (a pS (y,-x;)? by 6,,(1 -3} 
i=l i=l 


ss 
6, = max 


where B =(X'X) LX 'y,h, =x,(X'X) lx, and X=(x,,...,x,)!. 
The weighted least squares estimator of B is 


A ~ ae XY; 
B,, =A ? 
my 
il (6) + ,) 
where 
Hes aot 
T= (6, + 6;,) 


(y,-x,B,) =¥,- re (y,-x/B,). (7) 


sated Se 


Thus for empirical Bayes estimators, one has 


(y, - x;B,). 


0 herr 


Ooi 


4.2 The Conditional Mean Squared Error Estimator 


An explicit form for (3) can be obtained from the 
following formula for the derivative of the functions g, for 
empirical Bayes estimators, 

ag(y) 06, ag»), 
dy, dy, 


nd a 
06, OMe att (6 +6.) 


The partial derivatives appearing in (8) can be evaluated 
using standard methods. They are given by 


06° 5) 
ar (y, - xB), 
Oy ep) 


and 
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og.(y) 5 op 
. oy - x;B,) + + —x; - 
a6, (6° 0) (62 bm) 00,, 
where 
op, rane Ss XV = 1B). 
06° el. acre)” 


From (8), one has a closed form expression for mse , ({i,). 
This statistic is an estimator of mean squared error for the 
empirical Bayes estimator for the i-th small area with 
respect to the sampling design only. It is valid for any 
sample size n; it relies on the sole assumption that the direct 
estimators y, are normally distributed. When é =0, 
fi, =x; ibe and the derivatives in (8) simplify oe aerate 
Since 062/ dy, = 0, one has 


mse .(fi,) = (y, - x/B,,)? - Og lad o 


The properties of the conditional mean squared error 
estimator are best investigated in the simple situation where 
all the parameters of the smoothing model are assumed to 
be known. In this situation, dg,(y)/ dy, = -6,,/(6,,+6 o) 
and the conditional mean squared error estimator is equal to 
mse’, (fi,) = max {(mse ,(fi,), 0} where 


2 2 
6.,.0. O.. 
wv ii tn ,2 2 
=e {(y,-x1B) ~ 9,7 Sy \. 


0,,+ 9, 0,.+0 


mse ,(fi,) = 


The model based alternative to this estimator is the 
2 2 
posterior variance, 6,,0,/(6,, + 6), which coincides with 
EylEs {mse ,(fi;)}]. This estimator is a special case of 
Prasad and Rao (a 990) estimator and is denoted MSCpR (fi,). 
Estimator msec (fi, ) 1s highly variable when o 1S small. 
Indeed, when o? is close to 0, about 50% of the conditional 
mean squared error estimates are null. To further compare 
the 2 mean squared error estimators, conditional and uncon- 
ditional, observe that when all the parameters of the 
smoothing model are known, the conditional mean squared 
error of fi, is 
9,0, Cae 
E, {mse ,(fi,)} =——— + ; 
Sr 0. | 0.76 


(cn, =). -o7} 


The next proposition compares the average mean squared 
errors of the estimators, conditional or unconditional, of 
E.{mse ,(f1,)}. 

PROPOSITION 4: When o,,=0’, for i=1,..., and 
when the small area means are p1,’s are drawn using (6), the 
efficiency of the posterior variance with respect to the con- 
ditional mean squared error estimator for estimating the 
conditional mean squared error is 


E,{)) MSE.{mse ,(fi,)}/n] _ 0% +2076, 


vp MSE,,{msep, (fi,)}/7| o! 


Vv 


wl 


where MSE.(-) denote a mean Squared error taken with 
respect io hon distribution of the y,’s which are independent 
Niu,,6 *) random variables. 

The above efficiency is larger than 1 provided that 
6, iene Proposition 4 shows under heavy shrinking, 
the unconditional mean squared error estimator is a better 
estimator of the conditional mean squared error than the 
conditional estimator. This surprising result is caused by 
the large variance of the conditional estimator; when 
shrinking is extensive, it is a poor estimator. 

In some situations, such as that consider in section 5.1, 
shrinking is light and the use of the conditional mean 
squared error estimator is appropriate. The conditional 
efficiency of i, with respect to the direct estimator y, is 
given by o,, (mse; EAL). This i is larger than one provided 
that (y, ia 12] (6,,+6,)<2. Assuming that the 
smoothing model holds true, conditional efficiencies less 
than 1 can be expected for approximatively 16% 
(=P[N(0.1)?<2]) of the small area estimators. This per- 
centage should be higher if the smoothing model is 
deficient. Conditional efficiencies less than 1 occur in 
small areas having large residuals. On the other hand, the 
unconditional efficiencies, calculated with the posterior 
variance are, in this situation, less than 1 for all small areas. 
This shows that it is practically impossible for all the con- 
ditional efficiencies to be less 1; this had already been noted 
by Rao and Shinozaki (1978) for James-Stein estimators. 

Many of the observations made in the unrealistic situa- 
tion where all the parameters are known also apply when 
parameters are estimated. The unconditional alternative to 
the conditional mean squared error estimator is Prasad and 
Rao’s (1990) estimator, 


a2 t4- Tr a2 
ar poriots 0, x, ae o;, Var (6, ) (9) 
a septimar pia wulee = quaeeiinc an 
olen ome ae (od +6) (GG, ) 


aay oo) ES a2\2 Hs : ¢ 

where Var (6,) = iy (6,,+ 6,)°/n~. To investigate the 
extent to which Proposition 4 holds when parameters are 
estimated, a small Monte Carlo study was carried out along 
the lines of the approach ii) simulation study of Prasad and 
Rao ea In all the simulations, = 30 and o,, = 1, for 
Dans ani, he. smoothing model (6) was ul, = ue + v, and 
various values of o were used. The results reported i in 
Table 1 are based on m = 5000 Monte Carlo replications. 

The simulations used 5 sets of p-values whose vari- 
ances are reported in Table 1. For each set, y, was simu- 
lated repeatedly as a N(u,, 1) random variable, i = 1, ..., 7. 
The empirical Bayes estimate fi, was calculated for each 
small area and the mean squared error for small area 7 was 
calculated as MSE,=)>* (jfi,-p,)*/m where ))” 
denotes the sum on the m Monte Carlo replications. The 
efficiency of the empirical Bayes estimator for small area i 
is 1 /MSE,. The mean and the median of the n = 30 small 
area efficiencies are given in Table 1. The 2 mean squared 
errors, conditional and unconditional, were calculated for 
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each small area in the m Monte Carlo replications; from (9), 
MSepp (fi;) =(6 +5/n)/(1 +6) for each small area. 
Table 1 presents the mean and the median of their absolute 
relative biases, defined as [>> (mse (fi,) - MSE | 
(mMSE,) and of their coefficients of variation which are 
equal to (.* (mse, (fi,) - MSE, )?/m)!?/ MSE,. 


Table 1 
Relative Efficiency of the Empirical Bayes Estimators (RE), 
Absolute Relative Bias (RB) and Coefficient of Variation (CV) 
of two MSE Estimators (n = 30). All Results are Expressed in 


Percentage 
Yu, -f)*/29 RE% RB,% RBpp% CV,% CVpR% 

ils} mean PADD 1 47 97 51 
median 214 1 40 100 43 

2.53 mean 149 2 30 37 3] 
median 163 2 31 37 32 

Su mean 129 Z 20 23 20 
median 133 1 21 24 21 

4.24 mean 125 2 19 19 20 
median 131 : 1 22 20 22 

4.93 mean 122 1 iG/ 15 18 
median 133 1 17 13 17 


As shown in section 2, mse ,(fi,) is unbiased; the biases 
reported in Table 1 are caused by Monte Carlo errors. 
When n = 30, the condition o, */ 02> 2.4 derived in Propo- 
sition 4 for the conditional estimator to improve on the 
unconditional CN is not sufficient; the stronger 
condition o, */0*>4 is needed. Noteworthy is the fact that 
in Table 1, for YS (wu, -0)?/29>2.5, the CV of 
MS€pp (f1,) is only bias. Table 1 confirms that, when & iS 
of the same order of magnitude as o,, or smaller, the 
squared residual dominates the distribution of the 
conditional mean squared error estimator; in such cases 
Prasad and Rao (1990) unconditional estimator is a better 
estimator of conditional mean squared error. Even in 
situations when mse _,(fi,) cannot be recommended as an 
estimator for the conditional mean squared error, it still 
provides interesting diagnostic information: changes in the 
conditional estimators give a basis for comparing two 
smoothing models. This is illustrated in section 5.2. 


4.3. Conditional Mean Squared Error and 
Prediction Variance 


This section explores the relationship between the condi- 
tional mean squared error proposed in this paper and the 
prediction variance which is an unconditional measure of 
accuracy. Using the rotation of (6), the prediction variance 
is MSE(fi,) = £,,[E,{(fi, -x;B-v,)?}J. From the 
construction of presented in section 2, one has 


E,{mse ,({i,)} = EMA, - x; B - vP} 
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Thus we have the following result, 


PROPOSITION 5: The conditional mean squared error of 
empirical Bayes small area estimators satisfies, 


Ey|Es {mse ,(fi,)}] = MSE((i,), 


where MSE({i,) is the unconditional prediction variance. 


Proposition 5 shows that mse ,(fi,) can be looked at as 
an intermediate step in the evaluation of the unconditional 
mean squared error of fi,. Consider for instance the calcu- 
lation of Prasad and Rao (1990) o(1/n) approximation to 
MSE ({i,), 


2 igs 2 6) 
o.0 o.x.A'x 6 Var(6‘) 
A Tie: ‘ef q W v 
MSE prt) > cae) CRNERnASii OE Baie 
Gi, +9, 4 (G5 29,,) (6, + 0,) 
A 2 ° 
where Var (6,) =2> (o,, + 0,)*/n?. The standard deriva- 


tion, as reviewed in section 3.2 of Singh, Stukel, and 
Pfeffermann (1998), is based on Kackar and Harville 
(1984). An alternative derivation, presented in Belmonte 
(1998, 1999), is to take the expectation of mse_({i,), 
obtained using (8), with respect to the marginal distribution 
of the y,’s, which are independent V (x; B, 6, + 6, =.) deviates 
and to retain only the higher order terms. 

Proposition 5 holds in situations where the small area 
estimators are bench-marked, or where corrections sug- 
gested in section 3 are implemented. These are cases for 
which there are no closed form formulas for the prediction 
variances. Proposition 4 suggests a simple method for 
constructing unconditional Monte Carlo estimates. It 
suffices “9 generate a large number of replicates of 
{y,,1=1,...,2} where y, follows a N(x; is 6, + 6,,) and 
to ies mse ,(fi;) for each one. Averaging the 
mse (fi,)’s gives a plug-in unconditional prediction 
variance, equal to the MSE of Proposition 4 evaluated at 
estimates B,,, & of the unknown parameters. Unfortu- 
nately, this estimate is biased (this is a first order estimate 
in the terminology of Singh, Stukel and Pfeffermann 
(1998)). For the empirical Bayes estimator given by (7), 
according to (9) the bias of the Monte Carlo estimate 
derived form Proposition 4 is -o;, *Var(é VEC. 46, ae 
Further work is needed for constructing, using Proposition 
4, unbiased unconditional prediction variance estimators. 


5. ESTIMATING THE UNDER-COVERAGE 
IN THE 1991 CANADIAN CENSUS 


In 1991, the under-coverage of the Canadian Census was 
estimated using two surveys, the Over-coverage Study, 
which estimates the number of persons double counted or 
erroneously counted in the Census and the Reverse Record 
Check (Burgess 1988) for the persons missed in the Census. 
Combining these figures gives estimates of the under- 
coverage of the Census. This section investigates several 
estimators of census under-coverage. 
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5.1 Provincial Estimations 


The 1991 under-coverage rates for the ten Canadian 
provinces and the two territories with their coefficients of 
variation, expressed in percentage, are given in Table 2. 
The proportion p, of the population living in each province 
(the word province is used in this section to denote the 10 
Canadian provinces and the two territories) is also pro- 
vided. The coefficients of variation (CV) of Table 2 were 
calculated from variances estimated with 5 random groups. 
Thus, one can consider that the sampling variances have a y 
distribution. Throughout this section, we assume that the 
provincial under-coverage estimates and their variances are 
independent. 

Several estimators for provincial under-coverage are 
proposed by Royce (1992). Rivest (1995) proposed a 
composite estimator that shrinks the provincial under- 
coverage rate towards the national rate. It is given by: 

fOr = Ore 
where r,, = Ne pr, 1s the national under-coverage rate and 
the shrinking parameter @ is given by: 


pM ani —Tnu 
Ae! - p,)e; a pra ~Tnu 


This is the value of o that is optimal for loss functions for 
the estimation of provincial totals and of provincial shares 
of the population; see Royce (1992) and Rivest (1995) for 
details. One has 7; =r, + g,(r), where 


a= 


> 7,1 -p,)o, 
Dep -p,)o, ag aaa i; hie 


A closed form expression for the conditional mean square 
error estimator can be calculated easily by noting that 


A eae (7, — Ty). 
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The second partial derivative of g,(r) can also be calcu- 
lated; it has the same sign as r,—ry. Thus positive 
skewness in the under-coverage rate, that is likely when 
estimating rare events such as being missed by the census, 
increases the conditional mean squared error in provinces 
where the under-coverage is above the national rate. 

For 1991, @ = .874 and the national under-coverage rate 
Is r, = 2.872%. Table 2 gives the provincial composite 
under-coverage estimates, 7;° together with their efficien- 
cies eff;, = o,,/mse,(r,°), where mse ,(7,°) is calculated as 
defined in section 2, with the correction proposed in section 
3.2 to account for estimated variances. The composite 
estimator is an improvement over the direct estimators in all 
cases except three, that correspond to the provinces with the 
most extreme under-coverage rates. 

Table 2 also gives the empirical Bayes estimator r? 
calculated with a location smoothing model. Under model 
(M), the true under-coverage rate @. is assumed to be 
SLE: as 2 N(B, 6 : The parameter estimates are 

=1.45°% 10° and 6. = 2.61%. Two efficiencies with 
Bf to direct estimators are presented, eff? which is 
calculated with the conditional mean squared error esti- 
mator for r, ie including the adjustment of section 3.2 to 
account for estimated variances, and eff ie which is 
calculated with Prasad-Rao unconditional estimator. The 
large under-coverage rate in the N.W. Territories is respon- 
sible for the large estimate for 6; this makes the empirical 
Bayes estimators rp much closer to the direct estimators r, 
than the composite estimators. Redoing the analysis without 
the N.W. Territories and Yukon changes the empirical 
Bayes estimates drastically. 


Table 2 

Two Estimators of Provincial Under-Coverage and Their Efficiencies 
PROVINCE D, a CV oe eff; 2 eff, effion 
Newfoundland 2.06 1.994 15.96 2.105 Nei 2.038 1.07 1.04 
Prince Edward Island 0.47 0.931 30.00 1.176 0.65 1.025 0.93 1.03 
Nova Scotia 3.26 1.889 20.05 2.013 1.11 1.959 1.09 1.06 
New Brunswick 2.66 8245 A13e/3: 3.198 1629 3.162 1.14 1.09 
Québec PN, 2.605 8.35 2.639 1.16 2.605 1.04 1.02 
Ontario 37.24 3.641 8.46 3.544 0.87 Spo 1.02 1.04 
Manitoba 3.96 1.86 20.83 1.987 1.10 1.936 1.09 1.06 
Saskatchewan 3.58 1.798 18.87 1.933 1.04 1.863 1.06 1.05 
Alberta 9.24 1.995 14.57 2.106 1.01 2.032 1.06 1.03 
British Columbia 12.01 DISs3} 9.86 Drip 1.26 PRIA) 1.07 1.03 
Yukon 0.10 3.83 15399 3.709 27 3.56 1.05 sey) 
N.W. Territories 0.22 Sli ees 5.116 0.96 4.813 0.49 1.18 
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In Table 2, the composite estimator performs better than 
the empirical Bayes estimator; it provides gains in condi- 
tional efficiency larger than 10% in 7 of 12 provinces. 
Three efficiencies are smaller than 1; the discussion in 
section 4.2 suggests that efficiencies less than 1 are 
unavoidable. The relatively poor precision of 6,, (they are 
estimated using only 4 degrees of freedom), lowers the 
conditional efficiencies of the empirical Bayes estimators. 
It does not affect the composite estimator as much since it 
uses the same shrinking parameter for all provinces. The 
conditional efficiencies capture the poor performances of 
the r,;° and r,; in the provinces with the most extreme 
under-coverage rates. This is missed by the Prasad Rao 
efficiencies. They highlight the gains that smoothing brings 
to the two territories where the under-coverage rates are 
highly variable. The Prasad Rao efficiencies are meaningful 
only if one accepts the hypothesis of provincial exchangea- 
bility underlying the smoothing model. This is doubtful 
since under-coverage tends to be higher in large urban 
provinces than in small rural areas. 


5.2 Sub-Provincial Estimations 


Dick (1995) considered the estimation of the adjustment 
factors for census under-coverage for age x sex categories 
within each province for the 1991 census. The adjustment 
factor for a small area is defined as F=1+ (estimated under- 
coverage)/(census count). With four age categories, 0-19, 
20-29, 30-44, 45+, and two sexes, there are 96 small areas. 
The explanatory variables are interactions between the indi- 
cator variables for the 12 provinces, the 4 age groups and 
the two sexes, and the proportions of renters (R) and of 
people that do not speak either official language (L) in the 
96 small areas. In each one, the estimated variance was 
given by 6., = (under-coverage variance) / (census count)’. 

Dick (1995) regressed the log-variances on the census 
count to smooth the variance. He considered the expo- 
nentials of the predicted values for the log-variances (6, ) 
as the known variances. This underestimates the variability. 

Indeed, the average predicted variance G,, represents 
only 68% of the average unsmoothed variance. “Multiplying 
6,, by exp (62 / 2) = 1.54, where 6 is the residual variance 
of the smoothing model, corrects this problem. Fitting 
Dick’s (1995) model using the “unbiased” smoothed vari- 
ance yields & = (0. This is a degenerate situation where 
empirical Bayes estimators are equal to linear model pre- 
dicted values. Note also the correlation between the vari- 
ance residuals and the partial derivatives of g,, calculated 
as if om > 0, 1s, 0.25. This suggest that (5) is violated. 
Using 6, exp (6 /2) in the calculation is likely to over- 
estimate the precision the small area estimates. To illustrate 
the application of the conditional mean squared error 
estimator, these problems are ignored and the remainder of 
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this section assumes that the sampling variances o,, are 
known and equal to their smoothed values G,,. 

The model fitted by Dick (1995) has ten independent 
variables; the weighted least squares estimates and their 
standard errors, given by the square roots of the elements on 
the diagonal matrix of ye , appear in Table 3. The condi- 
tional mean squared errors mse, ({i fi,) for the 96 small areas 

can be calculated using (8). One ‘had mse a(t ) =O and 
mse, ((i fi,)>6,, for respectively 51 and 15 small areas. The 15 
small areas with large conditional mean squared errors need 
special attention: can the prediction model be improved for 
these areas? Two systematic features among the 15 corres- 
ponding residuals are noteworthy: there are 2 large positive 
residuals in the M/O-19 category and 2 large negative 
residuals in the F/45+ category. This suggests adding M/0-19 
and F/45+ as independent variables. The additional column 
to the X matrix for M/O-19 contains 1’s for the 12 small areas 
for males between 0 and 19 years old and 0 elsewhere; that 
for F/45+ is constructed in a similar way. Only F/45+ 
improves the fit; adding this explanatory variable gives the 
modified Dick model of Table 3. The absolute value of the 
t-statistic for F/45+ is 3; this is clearly significant. 

It is interesting to compare the conditional mean squared 
errors obtained with the modified Dick model with those for 
Dick’s model. Using the modified model decreases mse, 
in 26 small areas and increases it in 21; showing a slight 
improvement with the modified model. 

The sub-provincial empirical Bayes adjustment factors 
can be aggregated at the provincial level. Provincial adjust- 
ment factors is are given by 


Ee 
Foz 


Paya 


where C;, represents the census count for the i-th small area 
and pe is the summation over the 8 small areas in 
province p. A mean squared error for the provincial 
adjustment factor, either conditional or unconditional, can 
be calculated using a mean product error matrix mpe as 


Le » C; C,mpe(F,,, F'), 


1 
ey 


Conditional mean squared errors are obtained by using 
formula (2) for mpe. Lahiri and Rao (1995) give a formula 
for the off-diagonal terms of the unconditional mean 
product error matrix whose diagonal is given by Prasad Rao 
(1990) mean squared errors. 


mse (F’,) = 
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Table 


3 


Two Linear Models for Small Area Correction Factors: 
Dick ( p=11) and Modified Dick ( p=12). Parameter Estimates 
are Given With Their Standard Errors in Parentheses 


Category Variable Dick modified Dick 
mean intercept 1.0076 (0.0018) 1.0099 (0.0018) 
Age* Sex Interaction M / 20-29 0.0563 (0.0038) 0.0541 (0.0037) 
M / 30-44 0.0207 (0.0036) 0.0185 (0.0035) 
F/ 20-20 0.0243 (0.0038) 0.02223 (0.0037) 
F/ 45+ - -0.0102 (0.0037) 
Province* Renters Interaction BC*R 0.0436 (0.0115) 0.0433 (0.0110) 
Ontario*R 0.0791 (0.0100) 0.0789 (0.0102) 
Québec*R 0.0253 (0.0097) 0.0259 (0.0090) 
N.-B*R 0.1039 (0.0194) 0.1032 (0.0186) 
Yukon*R 0.0633 (0.0179) 0.0634 (0.0175) 
NWT*R 0.0687 (0.0117) 0.0680 (0.0285) 
Language*Sex*Age Interaction L*F/0-19 0.0802 (0.0293) 0.0680 (0.0285) 
Variance 3.368 1e-05 (2.45e-05) 2.21e-05 (2.30e-05) 


oS 


Table 4 
Direct (F,) and Empirical Bayes (F Be) Estimates of the 
Provincial Correction Factors With Their Conditional (eff_) 
and Their Unconditional (eff pr) Efficiencies. A Conditional 
Efficiency is e When the Conditional Mean Squared Error 
Estimator is Null 


b 


PROVINCE F F eff eff, 


P P pe DPR 
Newfoundland - 1.0203 1.0176 6.49 2.94 
Prince Edward Island 1.0094 1.0153 1.03 4.52 
Nova Scotia 1.0193 OME 5:3 2.59 
New Brunswick 1.0335 1.0367 0.67 bail 
Québec OZ6SI 1 O2620 * Inl2 0.93 
Ontario 1.0378 1.0396 0.68 0.93 
Manitoba 1.0190 LOLMI6) ~< 2.46 
Saskatchewan 1.0183 1.0166 ~~ 2.54 
Alberta 10204 MILOUS Ta 7-37 1.98 
British Columbia 1.0281 1.0293 1.09 1.03 
Yukon 1.0396 1.0400 1.41 Si 
N.W. Territory ROS75a0 105505 140 132) 


Direct and empirical Bayes aggregated estimates are 
presented in Table 4 with two efficiencies. The empirical 
Bayes estimates retain much of the interprovincial diffe- 
rences. This suggest that the explanatory variables of the 
smoothing model have captured most of the differences 
between the provincial under-coverage rates. A notable 
exception is Prince Edward Island’s small correction factor 
which is not accounted for by the explanatory variables. 
This is the only province for which the two efficiencies 
differ substantially. The conditional efficiencies are more 
unstable than the Prasad Rao efficiencies. Except in Prince 
Edward Island, both tell similar stories: in New Brunswick, 
Quebec, Ontario, and British Columbia, the aggregated 
empirical Bayes estimates do not improve much on the 
direct estimators. 


6. CONCLUSIONS 


The estimator of the conditional mean squared error 
proposed in this paper has several interesting features. It 
can be implemented with any shrinking strategy. It is con- 
ditional on the realization of the smoothing model used to 
produce the small area characteristics; thus the conditional 
estimator has a large sampling variance. Simple modifi- 
cations to the estimator are available to handle skewness in 
the data and estimated variances. In an empirical Bayes 
setting, it provides diagnostic information concerning the 
smoothing model. It can also be used as building blocks for 
estimators of the prediction variances when this variance 
has no closed form expression. 
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APPENDIX 


Proof of Proposition 1 
Let =’? be a symmetric square root for Z, such that 
(=!2)2=y and z=Z7!?(y-p). Note that z has a M (0, J) 


distribution. In terms of the random vector z,E{(y-p) 
g()'} =p Bize + D!2z)}. Now the conditional expec- 


tation of z,g(u PY MED elven (ere Zesty, yt Sets 
equal to 
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i z,exp(-z, /2) 
R 


y2n 
Integrating by parts shows that the above integral is equal 
to 


g(u + miezdce 


[esses 12) exp(-z; /2) dg;(H +o) 
Ri Wi2x 02, 
Observe that 


ORAL ye tec ) eee hy 
rape eon 71 = >, a g, (p + p22), 


Since ©! is symmetric, ,’ = ,°. Thus the above 


expression is the scalar product between e, =!”, the i-th 
row of X'” (e, represents a nx 1 vector of 0’s except for 
the i-th component which is 1), and Vg(p + PONOZ)e,, the 
j-th column of Vg(y), evaluated at y =p + 5"z. We have 


1/2 : 
Etz,g(u to) Al a Baa ag ie tons a = 
ty 12 
gaye EWVg(u +y PATA ey REE oa 


This equality also holds eet ager BiZ.g (+) 2) 
=e, YI? E{Ve(p + yiz)}e,. In other words, 


E{zg(p a yiez)} Z YZ E{Ve(u su eraye 
This completes the proof. 


Proof of Proposition 2 


. Let E, denote the conditional expectation with respect to 


Vie SVN (Ys Vo Vi preed,) AO LY) = 2,(V 5 10r 
fixed values of (),, +) ¥j-y>Vjsy> +0 Y,)- One has 


EA(y, - u,)A(y)} = fe - A(t) f (t)at. 


To evaluate this expression, one can integrate by parts. 
Integrating (¢ - p,)exp { -(t - p,)’/(20,,)}/(2n0,,)'” in the 
above integrand yields 

gl? 

Ey, ~ Bj) A(y)} = 6, £{h' (y)} + = 


= 2 " f 
<Jnof! 4) che ts 
5 10) 


(210, ) 1/2 


i 


where h'(t) is the derivative of h(t). Repeated integra- 
tions by parts show that 
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'? 2 bl 2 5) 
| Ozmmper u,)” exp {(t Lee 


(2n0,,)'” 


exp {(t-p,)’/ (20,,)} ; 


(216, oe 


- rae - 1) +A} 


exp((t-H,)°/20,)) 


(2n0,, 4 


- | (o,h"(t) +h(1)) 


where h” (t) is the second derivative of h(t). This yields 
EOP yA) = 


Oj; Pj 


6, £,{h'(y,)} + E,{h" (y,)} + 0(p,). 


Taking, on both sides, the expectation with respect to the 
distribution of (),, ..-, ¥;_1>¥j.4> «+» ¥,) completes the proof. 


Proof of Proposition 3 


Let E, denote the expectation taken with respect to the 
distribution of 6,,, given all the other random quantities 
(5.6.59 d) 1D ‘is context one can write (dg,(y))/(Oy;) = 
h 6,5, where / is a function possibly depending on 
(y, 6 i? ,j #i). A Taylor series expansion of h gives: 


AG) = h(G,) + A’ (6) (6; ~ 9) 
A 2 
Si-O; 
ong Soe + O(6, - 6,)°). 
Since - )6,,/ 6; follows; (a x, a distribution, 
E Git G, ae = ‘202 1 (k - 1), and the centered moments 


of higher orders are O(1/k?). The above expansion 
reduces to, 


6,,E, {0g,(y)/dy, } =6,,h(6,,) +h" (G,, = 


ie} 


+ OCR?) 


It is clear that the bias of 6,,4(6,,) as an estimator of this 
expression is O(1/k), provided that h'(o,,) #0. One has, 
neglecting O(1/k*) terms, 


VG. 
ole n( DS 
Keel 
orly: Ussrigd)) Ge 
= 6,h(6,) + h'(6,,)E,)6,,| ——— - 6, 
ee 


h'(o,) _}, ( &-é, 
+——— FE \G. | ————__ -o 
9) 1 iW k+1] 


Survey Methodology, June 2000 


Elementary manipulations show that, in the above formula, 
the coefficient of h’(o,,) is null and 


efAC Rabo)! ot ns|clooviel gp 
ENG bie Oy =2 
k-1 


+ O(1/k?). 
k+1 
| = 6,E,{0g,(y)/dy,} + O(1/k*). 


This shows that 


jaw Brom 
B,\6,h| 
I u Ji || 


The proof is completed by noting that this equality holds for 
the unconditional expectation, taken with respect to the 
joint distribution of (y, 6,,,i = 1, ...,7). 


Proof of Proposition 4 


The mean squared error of the posterior variance as an 
estimator of the conditional mean squared error has only a 
bias term, 

o \* 2 
tay2 

——— | {(u,-x,B)?-9, , 

2 2, 

Git 6, 
while the mean squared error of mse.(fi;) has only a 
variance component which is given by 
4 
0. 
2 tp 2 
a Var . { (; - x; B) } 


GO; +0) 


2 4 
=|" —] {207 + 4(n, -x;'B)°o?). 


2 
OF 26. 


The efficiency reported in Proposition 4 can be evaluated as 
the ratio of the 2 average mean squared errors defined 
above. It is given by, 


204 + 40? S> (u, - x,'B)?/n 
Dalayee 8) 5.) 10 


Taking expectations of the numerator and of the denomi- 
nator. with respect to model (6) yields the result. 
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Cold Deck and Ratio Imputation 


JUN SHAO' 


ABSTRACT 


Imputation is a common procedure to compensate for nonresponse in survey problems. Using auxiliary data, imputation 
may produce estimators that are more efficient than the one constructed by ignoring nonrespondents and re-weighting. We 
study and compare the mean squared errors of survey estimators based on data imputed using three different imputation 
techniques: the commonly used ratio imputation method and two cold deck imputation methods that are frequently adopted 
in economic area surveys conducted by the U.S. Census Bureau and the U.S. Bureau of Labor Statistics. A cold deck method 
imputes a nonrespondent of an item by reported values from anything other than reported values for the same item in the 
current data set (¢.g., values from a covariate and/or from a previous survey). Although sometimes a cold deck imputation 
method makes use of more auxiliary data than the other imputation methods, it is not always better in terms of the mean 
squared errors of the resulting survey estimators. In a simple case we compare explicitly the mean squared errors and discuss 
situations under which one method is better than the other two. In general cases we propose to compare mean squared errors 
empirically based on some consistent estimates of mean squared errors. Estimation of mean squared errors of survey 
estimators in the presence of imputed data is itself an important problem in surveys. A numerical example related to the 
Transportation Annual Survey is presented for illustration. 
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KEY WORDS: Complex survey; Mean squared error; Nonresponse; Simple random sample; Variance estimation. 


1. INTRODUCTION 


Imputation is one of the most common procedures to 
compensate for nonresponse in survey problems. In addi- 
tion to many practical reasons for imputation, imputation 
using auxiliary data may produce estimators that are more 
efficient than the one constructed by ignoring nonrespon- 
dents and re-weighting. Suppose that we have a sample s 
selected from a finite population P consisting of some units 
represented by 7 = 1, ..., M, and that we observe {y,, ier} 
(respondents), rcs. Suppose also that we have auxiliary 
data x,’s observed for all i¢s and x,>0. The commonly 
used ratio imputation method (see, for example, Kalton and 
Kasprzyk 1986) imputes nonrespondents as follows. First, 
we create K imputation cells P,, a pel ee we cola 
according to a categorical auxiliary variable (which is 
observed for every ies and is typically different from x) 
such that for every k, the following model is assumed to 


hold: 


. aie 
y, = B, x, +x; “e;, 


ve 5 (1) 
P(a, = 1 | y,,x,) = P(a, = 1 | x,), 
where B, is an unknown parameter, e, is independent of x, 
with E(e,)=0 and unknown V(e,)=0,>0, a, is the 
indicator of whether y, is a respondent, and (a,,x,)’s are 
independent. Then, within imputation cell k, a 
nonrespondent y, is imputed by B, x,, where 


Bee Wy, (Wie (2) 


ier, ier, 


1 


is the best linear unbiased estimator of B, under model (1), r, 
is r restricted to the k-th imputation cell, and w, is the 
survey weight associated with the i-th sampled unit. Note 
that model (1) consists of a regression model between y, 
and x, (with no intercept and with error variance pro- 
portional to x,) and a response model which assumes that 
the response mechanism is independent of y,’s, given x,’s. 
This response mechanism is termed as missing at random 
by Rubin (1976) or unconfounded response mechanism by 
Lee, Rancourt and Sarndal (1994). Based on the imputed 
data set, the Horvitz-Thompson (HT) estimator of Y, the 
population total of y,’s, is 

Fe E( 2 wires Dd Bi) (3) 

k ier, iES, - Ty 

where s, is s restricted to the k-th imputation cell. The HT 
estimator of Y obtained by ignoring nonrespondents and 
re-weighting within each imputation cell is 


GCN SDB es 4-0 (Ew /D mi (4) 
k ier, ES, ier, 
It can be seen that if x, = 1,then the estimators in (3) and (4) 
are the same. Both estimators are unbiased if model (1) 
holds. (Throughout this paper, the bias and variance are 
with respect to model (1) and repeated sampling, unless 
otherwise specified.) Under model (1), however, ee iS 
more efficient than y if the size of r is substantially 
smaller than the size of s. Even if the regression model in 
(1) does not hold, ¥ p may still be more efficient than Y,, in 
terms of their mean squared errors with respect to repeated 
sampling (Cochran 1977, Chapter 6) when the response 
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probability is a constant in any given imputation cell (which 
ensures that ee and ses, are approximately unbiased with 
respect to repeated samplings 
The purpose of this note is to compare the efficiency of ee 

with other estimators of Y based on data with nonrespon- 
dents imputed by using a method called cold deck. A cold 
deck method imputes a nonrespondent of y-variable by 
reported values from anything other than y-values (e.g., 
values from a covariate and/or from a previous survey). 
Cold deck imputation is opposite to hot deck imputation in 
which a nonrespondent is imputed by a respondent from the 
same variable in the current survey. The ratio imputation 
method uses both reported y-values and auxiliary data and 
is sometimes called a “warm deck” method. The simplest 
cold deck imputes a nonrespondent y,, ies - r, by x, and 
the resulting HT estimator of Y is 


Vee) ele ae (5) 
(KS iGSar 

The use of this simple cold deck is motivated by the fact 
that under model (1), B,’s are close to 1 in many survey 
problems, especially when x,’s are y-values Ue a previous 
survey. When some B,’s are ‘not equal to 1, Y, in (5) hasa 
bias which does not vanish even if s = P (i.e., Ai sample is 
a census). However, having a small bias may be paid off by 
lowering the variance so that the overall mean squared error 
mse (Y,, J=E (¥,- -yy may still be smaller than the mean 
satan error mse(f, y= E(Y, - a V(¥, - Y), where E 
and V denote the expectation and variance tinea model (1) 
and repeated sampling. More details can be found in section 
2. The simple cold deck may be improved by another cold 
deck method, the cold deck-ratio method, which imputes a 
nonrespondent y, by x, 7,/X,, where ¥, and x, are reported 
values from a previous survey. The corresponding HT 
estimator of Y is 


Vea =wyi+ YS wx 7, 1%,. (6) 
ier 1€S-Fr 

The estimator in (6) is unbiased if model (1) holds for y, 
and x, (7.e., y, = B,x, + ca é,) with the same B, as the one 
for y, and Xi These two cold deck methods are widely used 
in economic area surveys conducted by the U.S. Census 
Bureau (King and Kornbau 1994) and the U.S. Bureau of 
Labor Statistics (Butani, Harter and Wolter 1998). 
Applying cold deck imputation methods does not require 
knowing the imputation cells, although model (1) is 

assumed to ensure the unbiasedness of Be and ¥; 
Although the cold deck-ratio method rmkees use sof more 
auxiliary data, it is not always better than the simple cold 
deck or the ratio imputation method. In section 2 we 
compare explicitly the mean squared errors of Yy os Vg c and 
Y._, in a special case where the sample s is a simple 
random sample (SRS) and the response probability is a 
constant. Situations under which one method is better than 
the others are discussed. If the sampling design or the 
response mechanism is complex, then it is not easy to 
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compare the mean squared errors explicitly. One may, how- 
ever, estimate the mean squared errors of ie YF and ven 
and make an empirical comparison. Varance or on 
squared error estimation is itself an important problem, 
since it is common to report variance or mean squared error 
estimates along with the estimated totals. These are 
discussed in section 3. 

Our results can also be applied to the problem related to 
two-phase sampling or double sampling, which is often 
employed when it is cheap to take a large sample {x,,i€s} 
and expensive to obtain y-values so that a subsample 
{y,,i€r} is taken in the second-phase, rcs. 

A numerical example is discussed in section 4 using data 
from the Transportation Annual Survey conducted by the 
U.S. Census Bureau. 


2. SRS WITH UNIFORM RESPONSE 


To illustrate the idea, we start with the simplest case 
where s is an SRS (without replacement from P but the 
sampling fraction is negligible); there is only one imputa- 
tion cell so that we can drop the subscript k for imputation 
cell; and the response probability is a constant p>0O 
(uniform response mechanism). 

In this case w, = N/n, where n is the size of the sample 
s and N is the size of the population P. Since n/N = 0 is 
assumed, 


ee Ps (7) 


for large n, where p. = E(x,) and v, = V(x,) and, through- 
out the paper, A = B means that A is equal to B up to a term 
which is relatively negligible compared to A and B as all 
sample sizes in imputation cells increase to infinity. A more 
detailed derivation of result (7) is given in the Appendix. 
For ¥ wy in (4), it is easy to see that w, = N/r, where r is the 
size of r, and Y,,, is unbiased. Then 


is = = oh, a 
P P 


Hence Ne is more efficient than 1a unless p=1 and 
Bev, =0. The gain in using Y,, is proportional to B? and v, 
both are measures of resist of the auxiliary vaniable x a 
in explaining y through model (1). 

For the simple cold deck, 


Caen Pe? ps: » “FT (bse Ba oe *) 


ier ie€s-r ier jer iés-r 


mse (¥,,) = V(¥, - Y) = VY, 


where e,’s are defined in (1). Consequently, 
5 N2 
ACO ——{o°Pn, + (B’p+1-p)y, 


+ (B-1)°p(1 -p)n?} (8) 


Survey Methodology, June 2000 


(see the Appendix). The bias of els 
E(¥, - ¥) = Nu,(1-p)(1 -B) 


and, hence, 


mse(¥,.) = V(¥. =) Os [E(¥., eee 


«V(¥,) +[E(¥.- YP 


N2 
7 ——{o°Pu, + (Pip+1-p)y, 


+(B-1)°(1 -p)[p+n(1 -p) ui} (9) 


Comparing (7) and (9), we obtain the following 
conclusions. 


1. When p=1 (no nonresponse), mse (Y,.) = mse (Y’,). 


2. When p<1 and B = 1(y and x have the same mean), 
mse(Y..) < mse(Y,). 


3. When p<1 and B#1, mse (Y,.) < mse (Y,.) if and only if 
(B-1)°[p+n(1 -p)|p, +1 -B?)v,/p, 


-0’(p+1)/p<0. (10) 


Assume that u,>0. In most economic surveys, the 
relative variance v,/ 2 is smaller than p + n(1 - p). 
Hence the left hand side of (10) is a quadratic function 
of B with a positive coefficient in the B* term and, there- 
fore, the simple cold deck is better when B is in the 
interval with limits 


[p+n(1 -p) In, tyvy/ uz + (Ip +1 -p) In, -v,/ He (P+ D/p 


[Ip +n(1 -p)]y, - v,/B, 


This interval contains | since (10) holds if B = 1. Note 
that [p + (1 - p) J, increases to infinity as 1 increases 
to infinity. Hence the interval of B’s for which the 
simple cold deck is better shrinks to a single point 
(B =1) asn-~-, 


We now consider the cold eenouete Assume that 
= Be, +x,2,, E(,)=0, VE,)= 07, and that é,, e,, and 
6.5) are mutually independent Let 2, =x SIS x, and 
Gla oh eye ae >. Then E(Y,, ap Oye and 


4 Ve 2 2, Dia) oe 11 
mse(Y,. ,) = —{o pu, +B v,+o (1 -p)y,f, (11) 
n 


where y,. = E(x; /%,) (see the Appendix). By (7) and (11), 


mse (Y,,) -mse(¥,. p) Sees - } L. | (12) 
n P 


and, hence, the cold deck-ratio is better than the ratio impu- 
tation method if and only if 1/p+12> y,/p,. Note that 
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¥, 2, and y, is close to p, if x, and x, are highly and 
positively related, in which case cold deck- ratio imputation 
can be much better than ratio imputation. 

The comparison between the simple cold deck and the 
cold deck-ratio is the same as that between the simple cold 
deck and the ratio imputation method. One only needs to 
replace (p+1)/p in the third term of the left hand side of 
(10) by y, /p,. 

The parameters B, 6, u,, v, and y, have to be estimated 
in order to compare the efficiencies of Kes Ve and ie 
Instead, we can iene compare were eal mean aes 
errors of ie yy and Yes . This is discussed next. 


3. STRATIFIED SAMPLING WITH 
UNCONFOUNDED RESPONSE 


We consider the following stratified sampling design 
adopted by many U.S. government survey agencies: the 
finite population P is stratified into H strata with N, units 
in the h-th stratum; 7, >2 units are selected withoae repla- 
cement from stratum h, according to some probability 
sampling plan; and the units are selected independently 
across the strata. 

The survey weights w,’s are constructed so that if all 
y,’s are observed, the HT estimator Mies: y, is unbiased 
for Y under repeated sampling. 

We assume model (1). The response probability is no 
longer a constant, but independent of the y-value. For the 
cold deck-ratio, we also assume that within the , k-th 
imputation cell, ¥,=B,%,+¥)"é,, E(é,) = 0, VEE, a 6 and 
7) €.5 Oey) are mutually iaieeniiert 

Explicit results for the mean squared errors such as (7), 
(9) and (11) are not easy to obtain. We may, however, make 
empirical comparisons of the efficiencies of ay Ye and 
Y ek based on their estimated mean squared errors. 
Estimation of the mean squared errors of re if and Y, algxt 
is in fact an important part of the sampling aheory: It is well 
known that for imputed data sets, the naive method that 
applies the standard variance estimation formulas by 
treating imputed nonrespondents as observed data leads to 
underestimation. When no correct method (for estimating 
the mean squared error) is available, the naive method is 
used in many survey agencies. 

We now derive estimators for VY ) or mse (Y ) that are 
correct under model (1), where Y denotes Hy, Ve or Py 

Let E and V,, be the expectation oad, oan. in 
respect to model (1) and let £, and V, be the expectation 
and variance with respect to repeated sampling (conditional 
on the model and response). Then 


VEE (i OOP ey ark: (13) 


We first consider E_, [VY )], the first variance component 
mi 13), It suffices to obtain an estimator of VY ) 
conditional on {y,,x;,4,, ieéP} (and {y,,x,, ie P} for 
cold deck-ratio), where a, is the response indicator for y,. 


1? 
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The estimation of VY, c) and VAY, cp) is simple 
(which is an advantage of using a cold deck method). Let 


2 
: bs (ie Dy, vs (14) 


n,,~ 1 iesch) Nj, ies(h) 


be the standard variance estimator for )’,..w,t,; when 
{t,,i€s} is treated as an observed sample (from 
{t,,i i€P}), where s(h) is s restricted to stratum h. Then 
V (¥.) can be estimated by using (14) with ¢,=a,y,+ 
al ~a.)x, and yore _p) can be estimated by using (14) 
with t, =a.y,+ (1 - a)x,5 1, 

The estimation of VY. p) is slightly more complicated 
but similar. Assume that in each imputation cell, the 
number of sampled units is large and the response 
probabilities are bounded away from 0. Note that 


rE |(Lwai/E wa 


IES). Iery 


x Dina Bixs)+ +B, >> w,x, 


ier, iESy. 
- » a w,a,(y; ~ B,x;) + Beds wa 
k ies, ies, 


= wa, ~ Bx,) + Bx) 


ies 


where C, = EQ rc XE (vier, WX) and G = ¢, and B, =B, 
C LES. Aftér estimating B, by B, and ¢ by 
= Dien Xi! ier, wx,, we estimate V,(Y,) by using (14) 

ene = Cay, -Bx,) + Bix ;» where G= et and ve B, for 
i€s, 

fom we discuss the estimation of V, [E, y ) =rh the 
second variance component in (13), it should be noted that 
Vahey (Y) - YJ/E,, [V.(Y)] = = O(n/N). This is because the 
variance of E. (Y) - ¥ (if it is nonzero) is typically of the 
order N, whereas the order of V, (Y) is typically N?/n and 
thus the order of E[V, (Y)] is N?*/n under some regu- 
larity conditions. Hence, in theory, it is not necessary to 
estimate V [EY ) - Y] if the sampling fraction n/N is 
negligible. However, the constant in O(n/ N) is unknown 
and, hence, one may still want to estimate V,, [E(Y ee 24 
in applications even when n/N is small. 

We now consider the estimation of the second variance 
omen in (13). For ees 


BY) ova Flay, +d a,)x,] - a 


oa ace 
Then, under model (1), 
VLE, (¥,)-¥] = 
£,|32 % 2 (1 - -a,)x,|+V,,| >> (1 -a,)(B, - 1x,]. 


ieP, ieP 


Shao: Cold Deck and Ratio Imputation 


If we estimate o; by 


t= aW,(y, - Bx? /Daw,x i Xj» 


ES). TES) 


then an estimator of VLE, (Y.) - Y] is 


Vie oy Ga) wx, + 
k 


i€5, 
N 2; 
yay : u,-— dd: Mil 2 
h 1, — liesth) Nj, ies(h) (15) 


1)x, and B, = B, for i€s,. 


where u, = (1 - a,)(B, = 


For y eR 


(Fe aks =e (1 -4;,)(y;- 


ieP 


x,y; /%;) 


and 
VE a - 4 r 


E, | ad d-a)x +> &D ( “ait 


k ieP, k icP, 


Hence V, [E, (Y c-r) ~Y] can be estimated by 


VoCeR 2218 Ou (1 -a,)w,x, +6 cos dl -a,)w,x; f%, » (16) 
TES, IES) 
where 
1= > wy, - 8," / Dwg, 
IES, ' TES) 
and 
B, * oe w3/D W, 2). 
1ES, IES, 
For tes 
E,(Y,) 4 y~)) [Za/ Eas pancnie i bso 
k ieP, ieP, ieP, ieP, 


and from Taylor’s expansion, 


V_[E.(¥p)- Y= 


Pa{¥ oid * x, > (1-a,) x, 


1eP ye IEP, ieP, 


[Yaa 


It can be estimated by 


a7 LS yo wx > (i a)w,., 


1€S,, ies, 


[Same i (17) 


IES) 


Finally, ve Rp and Fay are unbiased but er has a bias 


ya - B,)E,, (i -a,)x, 


ieP, 


? 
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which can be estimated by 


DPD AY x a;)W;X,. 


i€s, 


Thus, we obtain the following estimated mean squared 
errors: mse(Y, ) can be estimated by 


sé (P,) a ais OR” 
where v, R is obtained using (14) with ¢, = Ca,( Vee B.x,) + 
B.x 2 é = ¢, and B, = 6, forieés,, and v,, is given by (17); 
mse ( ¥.) by 


+) (1 - By) YS w.x, 


IES, - rT, 


NS 
mse (¥.) =v, + 


where v,~ is obtained by using (14) with 1, =a,y, + 
(1 - a,)x, and v,,, is given by (15); and mse (Y, aes can be 
estimated by 

mse (Pop) Paice. Y2caRy 
where v,~_p is obtained by using (14) with t,=a,y,+ 
(1 -a,)x,¥,/X, and v.._p iS given by (16). 

Under model (1) and the asymptotic settings in Krewski 
and Rao (1981), Rao and Shao (1992) or Valliant (1993), 
the derived mean squared error estimators are asymptoti- 
cally unbiased and consistent as all sample sizes in imputa- 
tion cell increase to infinity. 

For cold deck or cold deck-ratio imputation, the first 
term (v,. OF V\~_,) in the estimated mean squared error is 
the same as the one obtained by applying a standard 
formula (such as (14)) and treating imputed nonrespondents 
as observed data. For ratio imputation, applying (14) and 
treating imputed nonrespondents as observed data produces 
the following estimator of mse (Y’, )s 


y, 2 
Vie y, Mine a : De W,2,-— ry We (18) 
h N, } 7) iest) N,, ies(h) 
with z,=a,y,+(1 - a,)B,x;, which is different from the 


first term v,, in our estimator mse (Ye ) and, hence, is not 
asymptotically valid even if n/N is ae eile 


4. AN EXAMPLE 


We consider an example using a data set from the 
Transportation Annual Survey (TAS) conducted by the U.S. 
Census Bureau. 

The TAS is a survey of firms with one or more establish- 
ments that are primarily engaged in providing commercial 
motor freight transportation or public warehousing services 
in U.S. A stratified simple random sample is selected 
without replacement from employers contained in the 
Census Bureau’s Standard Statistical Establishment List. 
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The strata, which are also the imputation classes in this 
example, are constructed according to company’s size 
within each industry. 

There are various variables in this survey. We consider 
the estimation of the population totals of the current year 
annual revenue (y) in four industries. The variable y has 
nonrespondents. Three covariates without nonrespondents 
are considered: the current year annual payroll, the previous 
year annual revenue, and the previous year annual payroll. 
The sample size, response size for y, and the sampling 
weight in each stratum and industry are given in Table 1. 


Table 1 
Sample Sizes, Response Sizes, and Sampling Weights 
Across Industries and Strata 


: Response Samplin 
Industry Stratum Sample Size nie Weight” 

1 0 31 24 1.00 
1 14 6 12.43 

2 1] 7 8.91 

3 10 4 6.10 

4 11 6 ais 

5 16 1 2.70 

6 18 13 217 

2 0) 86 82 1.00 
1 8 2 32,9) 

2 13 10 9.85 

3 11 9 10.82 

4 12 10 6.08 

5 13 10 3.60 

3 0 38 30 1.00 
1 14 9) 87.91 

2 11 8 67.39 

3 13 10 44.48 

4 14 13 25.28 

5) 16 13 isye/ 

6 18 12 9.80 

7 15 1] 6.23 

8 NS 14 4.68 

9 40 33 2s 

4 0 28 23 1.00 
1 7 5 32.14 

2 13 6 16.75 

3 10 7 12.90 

4 14 1192 7.00 

5 13 9 6.18 

6 1] 7 4,70 

a 17 12 B31 

8 19 14 1.89 

9 Pip) 16 1.82 


First, we use the previous year annual revenue as the 
covariate x in simple cold deck imputation and ratio impu- 
tation. The current year annual payroll and the previous 
year annual payroll are used as y and x, respectively. For 
four industries and three imputation methods, Table 2 lists 
the estimated totals, the proposed estimated MSE’s for the 
estimated totals, the naive estimated MSE’s for the 
estimated totals (obtained by treating imputed values as 
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observed data), and the MSE ratios (the proposed estimated 
MSE over the naive estimated MSE). Note that the pro- 
posed estimated MSE is the sum of v, and v, for the ratio 
and cold deck-ratio methods or the sum of v,, v,, and the 
squared estimated bias for the simple cold deck method. 
Values of v, and v, are also included in the table. 


Table 2 
Estimated Totals and MSE’s When x = the Previous Year Annual 
Revenue, y= the Current Year Payroll, and x =the Previous Year 


Annual Payroll 
Method 
Cold Deck- 
Industry Estimate Cold Deck Ratio Ratio 
1 Total 5.31 x 10° 5.19 x 10° 5.42 x 10° 
vy, T1210 B46 10 260 x 10” 
v 1,39" TO"'2.50"x 10" 1.81, x 10° 


p) 
Proposed MSE 2.30 


Naive MSE _ 7.73 


<qlO S534 <0" 440 co 10" 
x0" 8.46.10" 246x100? 


MSE Ratio Zo 3.95 1.79 
2) Total 1.66 x NO GOS GT RTO 
v, 4.00 x 105 4.19 x 10" 5.57 x 10! 
Vv, 6:03) <al0? 2) 88. cl Os Gra 4exnl OS 
Proposed MSE 1.02 x 10'® 3.30 x 10'° 6.23 x 10" 
Naive MSE 4.00 x 10% 4.19 x 10 5.58 x 10'° 

MSE Ratio 2.54 7.87 Tei? 
3 Total 3540 S30 Bs Soea0e 
vy 1.32 x 10'® 1.80 x 10'® 1.94 x 10” 
V, 5.44) xall01°9 8.62) <a 0-9 6.77 x 102 
Proposed MSE 6.97 x 10'° 1.04 x 10" 2.62 x 10" 
Naive MSE_ 1.32 x 10'° 1.80 x 10" 1.87 x 10" 

MSE Ratio 5.27 5.80 1.40 
4 Total VOTES TAO Ta Oe 
vy, 2.11 x 10 2.14 x 10! 5.13 x 10" 
V5 3 Sil x<alO2 8:26) <allO> 5106s 10 
Proposed MSE 2.59 x 10'° 2.97 x 10'° 1.02 x 10'° 
Naive MSE 2.11 x 10'® 2.14 x 10'® 5.06 x 10" 

MSE Ratio 28) 1.39 2.01 


Next, to see the effect of using a wrong covariate in 
using the simple cold deck method, we repeat the previous 
computations using the current year annual payroll as the 
covariate x, and the previous year annual revenue and 
payroll as y and x, respectively. The results are reported in 
Table 3. 

The following is a summary of the results in Tables 2 
and 3. 


1. The simple cold deck method depends heavily on the 
choice of the covariate x. When x is the previous year 
annual revenue (Table 2), the difference among the 
estimated totals provided by three methods is negligible; 
in terms of the estimated MSE, the simple cold deck 
method is the best. However, when x is the current year 
annual payroll (Table 3), the estimates from the simple 
cold deck is obviously too low; in terms of the estimated 
MSE, the simple cold deck method is the worst, because 
of its large bias (shown in Table 3). 


Shao: Cold Deck and Ratio Imputation 


Table 3 


Estimated Totals and MSE’s When x = the Current Year Annual 
Payroll, y= the Previous Year Annual Revenue, and x = 
the Previous Year Annual Payroll 


Method 
Cold Deck- 
Industry — Estimate Cold Deck Ratio Ratio 
1 Total 4.49 x10? 5.19 x 10? 5.39 x 10° 
Bias -8.99 x 10° 
vy, 8.10 x 10'* 8.46 x 10 2.85 x 105 
V, 13 Saxs1O 2641 0PM 75) <10P 
Proposed MSE 1.03 x 10'° 3.49 x 10° 4.60 x 10!° 
Naive MSE 8.10 x 104 8.46 x 104 2.55 x 10!° 
MSE Ratio 12.68 4.12 1.81 
2 Total SOS WOM seh Xe IO tal STO 
Bias “126x110? 
im 4.36 x10" 4:19 x10" 5.74 x 101% 
Vv, 8.20 x 10% 1.48 x 10! 8.95 x 10% 
Proposed MSE 2.73 x 10'° 1.90 x 10'° 6.64 x 10'° 
Naive MSE 4.36 x 10% 4.19 x 10% 5.62 x 10'° 
MSE Ratio O25 4.54 1.18 
3 Total SaORX 103535100 3.47 x10" 
Bias Slog x07 
vy 1.25 x 10'® 1.80 x 10'® 2.30 x 10!” 
v, 4569 1028925 108 5:4 oc 10'° 
Proposed MSE 1.89 x 10’? 1.10 x 10'” 2.84 x 10” 
Naive MSE 1.25 x 10'° 1.80 x 10!° 1.83 x 10” 
MSE Ratio IS713 6.15 1.56 
4 Total LOGE 10122510220) x 10% 
Bias -1.35 x 10° 
Vv, 1.93 x 10'® 2.14 x 10! 5.84 x 105 
Vv, 2.67 x 10% 4.62 x 10% 3.07 x 10% 
Proposed MSE 4.03 x 10'® 2.60 x 10’° 8.92 x 10% 
Naive MSE 1.93 x 10'° 2.14 x 10'° 8.92 x 10" 
2.09 22 1372 


MSE Ratio 


2. There is no definite conclusion on the relative perfor- 
mance (in terms of the estimated MSE) of the ratio 
imputation method and the cold deck-ratio method. In 
this example, the cold deck-ratio is better for industries 
1-3, whereas the ratio imputation method is better for 
industry 4. Some scatter plots of the data (not shown) 
indicate that the correlation between x and x in 
industries 1-3 is higher than that in industry 4, which 
might be the reason for the difference in relative perfor- 
mance of the two imputation methods. See also the 
discussion after formula (12). 


3. The naive estimated MSE’s are much lower than the 
proposed estimated MSE’s and are too optimistic. For 
example, in Table 3, the naive MSE’s for the simple 
cold deck method are always smaller than those for the 
cold deck-ratio method, although we know that the 
simple cold deck does not work well in this case. In this 
example, v,/v, is not small because of some large 
sampling fractions. Since the naive estimated MSE is 
either equal to v, (for the cold deck imputation 
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methods) or not very different from v, (for ratio 
imputation), the underestimation in using the naive 
estimated MSE is mainly due to treating imputed values 
as observed values in strata with large sampling 
fractions (and ignoring the bias of the simple cold deck 
estimators in the case of Table 3). 
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APPENDIX 


1. Proof of (7): When n/N~0, V(¥,- Y)=V(f,). 
Then (7) follows from 


P(E] 


IES ler ies 


Bee 
NE Bee Ry 
n Pp ff 


V(P,) = wea 50 * ‘/| Sy » 


for large n, where the last approximate equality follows 
from the fact that conditioned on Xe 


LAOS eg ae {Dee 
2. Proof of (9): Under model (1), 
A 2 
VP.) = ea yx") : r( Byeixnt DO “| 
n? ier ier iés-r 
sedis ec + BV Sie +V Bae 
ne x i I 


ier lés-r 


+ 28Cov| yx, dv * 


ier l€s-r 
= "(pn +B [pv, + p(1 - p) ey] 
+ (1 -p)(v, + pu;) - 2BpU -p)by} 
=X (otpn, + (Pp +1-p)y, 


+(B- 1p - p)py}. 
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3. Proof of (11): Under the assumed conditions on ( Vij x) 
and (J, £,), 


i N2 
mse(¥c_p)=—> 40247 o> ] 
n 


ier ieés-r 


PADS «+33 


SEE 


*2Coy| Sree | 


i€r IES 


2 
-v {o’p(u, + 7,) + (B’v, + 0*y,) -20°py, } 
n 
N2 
= {opp +B°v, +07 (1 -p)y,}. 
= 


REFERENCES 


BUTANI, S., HARTER, R., and WOLTER, K. (1998). Estimation 
procedures for the Bureau of Labor Statistics current employment 
statistics program. Proceedings of the Section on Survey Research 
Methods, American Statistical Association. 


COCHRAN, W.G. (1977). Sampling Techniques. Third Edition. New 
York: Wiley. 


KALTON, G., and KASPRZYK, D. (1986). The treatment of missing 
data. Survey Methodology, 12, 1-16. 


KING, C., and KORNBAU, M. (1994). Inventory of economic area 
statistical practices. ESMD Report Series 9401, Bureau of the 
Census, Washington D.C. 


KREWSKI, D., and RAO, J.N.K. (1981). Inference from stratified 
samples: properties of the linearization, jackknife and balanced 
repeated replication methods. Annals of Statistics, 9, 1010-1019. 


LEE, H., RANCOURT, E., and SARNDAL, C.-E. (1994). 
Experiments with variance estimation from survey data with 
imputed values. Journal of Official Statistics, 10, 231-243. 


RAO, J.N.K., and SHAO, J. (1992). Jackknife variance estimation 
with survey data under hot deck imputation. Biometrika, 79, 811- 
822. 


RUBIN, D.B. (1976). Inference and missing data. Biometrika, 63, 
581-592. 


VALLIANT, R. (1993). Poststratification and conditional variance 
estimation. Journal of American Statistical Association, 88, 


89-96. 


ase 


re me - 


ioe 


a 


> rc 19 ae - 
arf 4 
{ ~ isin a 
~ \ on? * 
if . 
t Leis ¢ ald vy 
¥ pie wrens 
‘oa oe 
oO, See poiepdinistr a : 
sate ap ane 
a e a Parte 
(4-t CE it 
‘ a ~ 
ie al yy S = ‘ 1" 
' ~i te ts a 
rome : A ile ie 
< s . Wee : i 
‘ te Ot” Ser MNT of 
srrriengny ketipy oh i Ar acon hh pet 
a's ow Oh aes re i éis J 
ny bo 4 lg he ge i F 
eyed! vary . ogee ew, ; _ 
Peas 
at > (ora 43 id > iy onl vitae 
is a ¥ ve ay 3 MF 
' pao. hy Pate ROR) 0 pa oe bison fua sett r) 
« u@ 1068 ann? tae CIs aA, lode 4 tee, 4 
pe aewe Sc ieons 
Sipe | (eral ve + ae aes etl , ‘3 os in: fF { 
A: _ov lpeades®) 7a egy rete wy y 
Lae W oat awe, ethene preems s nt my “ q y 1 Gis Gue 
waged. e * TOUT eaAR LA Cr GOR we hs TA ee (fa, Hawn ish wi 
Uieech ter Age a Soeby De gia PENS. 2. eh = se steGian oe 
be eet ne tee Fe tar br kivipg Qaliagat reser oh UA ty atgaNeaM | wet 
: ; pine neato ‘Oneal a 
re i4y WO digi Wim, Viren yt lise ebeg Ss Wt aif shy he - 
> he MeN stir: Oe) wr al bediinaii Ae id 
“ ae "St 
ale Th 
iv seb: net geow \seatme * ; 
P Genas is ._ ene xed’ Gay AS ~~ 
perhaie tetra whem a) oe cat en po SoA Al 
ry neath Perey eat salgelad sce Oi, Hm tye = ee wf 
oe eh oe ae sascie, ai ai AY oe 
dh @ i oe ee ANN ae but hat 
vA olorna ida *¥ re ha s fo ae r. 


6 (Og 
Y 


( 


‘ We i. 
a 


Survey Methodology, June 2000 
Vol. 26, No. 1, pp. 87-98 
Statistics Canada 


87 


Model-Based Estimation With Link-Tracing Sampling Designs 


STEVEN K. THOMPSON and OVE FRANK! 


ABSTRACT 


Samples from hidden and hard-to-access human populations are often obtained by procedures in which social links are 
followed from one respondent to another. Inference from the sample to the larger population of interest can be affected by 
the link-tracing design and the type of data it produces. The population with its social network structure can be modeled 
as a stochastic graph with a joint distribution of node values representing characteristics of individuals and arc indicators 
representing social relationships between individuals. In this paper maximum likelihood estimators of population graph 
parameters are described. Predictors of realized population graph quantities are obtained using predictive likelihood. These 
estimators and predictors are compared with conventional data summaries and illustrated with a numerical example. 


KEY WORDS: Snowball samples; Adaptive sampling; Graph sampling; Ignorable designs; Link-tracing designs; Network 


sampling; Likelihood; Predictive likelihood. 


1. INTRODUCTION 


In studies of hidden and hard-to-access human popu- 
lations, link-tracing procedures, in which social links are 
followed from one respondent to another, are commonly in- 
volved in obtaining the sample. For example, in a study of 
injection drug use in relation to the spread of the HIV infec- 
tion, initial respondents may he asked to identify drug- 
injection or sexual partners who are then added to the 
sample. For such a study, the social links are of inherent 
importance for understanding the issues of interest while at 
the same time being useful or essential in building the 
sample. However, inference from the sample to the larger 
population or social structure of interest can be affected by 
the link-tracing procedures and the type of data they pro- 
duce. In this paper we evaluate this inference problem in 
relation to the design and describe some inference methods 
for such studies based on maximum likelihood estimation 
and prediction. 

Human populations with social structure are often 
modeled as graphs, with the nodes of the graph representing 
individuals and the edges or arcs of the graph representing 
social links, relationships, or transactions. The population 
graph itself can be viewed either as a fixed structure or as a 
realization of a stochastic graph model. In real studies of 
human populations, particularly those that are hidden or hard 
to access, it is seldom possible to obtain data on the whole 
population or graph structure. Rather, data are obtained from 
a sample, and the sample may have been obtained by 
innovative and unconventional means, including methods 
taking advantage of the arcs or links from one individual to 
another. The data may contain information about 
characteristics of sample individuals, social links within the 
sample, and in some cases information about links between 
individuals in the sample and individuals outside the sample. 


In this paper we use the term “sampling design” to refer 
to the procedure by which the sample is selected, whether 
deliberate or happenstance. For many ethnographic and 
sociological studies of hidden populations, link-tracing 
designs are considered the only practical way to obtain a 
sample large enough to study. In other studies, the social 
structure is itself the object of interest and the link-tracing 
methods are used in order to obtain meaningfully structured 
samples to study. 


The statistical literature on design and estimation with 
link-tracing designs includes procedures variously termed 
snowball sampling, chain-referral sampling, random walks, 
nexus sampling, network or multiplicity sampling, and 
adaptive sampling. A type of link-tracing design in which 
individuals in an initial sample were asked to identify a 
fixed number of acquaintances, who in turn were asked to 
identify the same number of acquaintances, and so on for a 
fixed number of stages or waves, was termed “snowball 
sampling” by Goodman (1961). A Bernoulli procedure was 
assumed for the initial sample. Snowball designs were 
developed in the graph setting with a variety of initial 
probability sampling designs and any numbers of links and 
waves by Frank (1971, 1977a,b, 1978a,b, 1979a), who 
obtained a variety of design and model based methods for 
estimating graph quantities from the sample data. Snijders 
(1992) used the same term “snowball sampling” to include 
designs in which only a subsample of links from each node 
is traced. The case in which only one of the links from a 
node is selected at random and followed to another node, 
and then one of its links selected, and so on, was called a 
“random walk” by Klovdahl 1989. Link-tracing sampling 
methods in which there is only one link from each node 
have been termed “chains” (Erickson 1979). Frank and 
Snijders (1994) consider model- and design-based 
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estimation of a hidden population size, that is, the number 
of nodes in the graph, based on snowball samples. Addi- 
tional practical and statistical issues in sampling from social 
networks with various types of snowball, chain-referral, and 
other link-tracing designs are discussed in Granovetter 
(1976), Morgan and Rytina (1977), Frank (1979b, 1981, 
1988), Watters and Biernacki (1989), van Meter (1990), 
Spreen (1992), Wasserman and Faust (1994), Spreen and 
Zwaagstra (1994), Karlberg (1997), Jansson (1997), Spreen 
(1998), and Robins (1998). 

Design-based estimation methods were developed 
additionally for the closely related designs of network or 
multiplicity sampling, in which social, kinship, and admini- 
strative links were traced (Birnbaum and Sirken 1965, 
Kalton and Anderson 1986, Levy 1977, Levy and 
Lemeshow 1991, Sirken 1970, 1972a, b, Sirken and Levy 
1974, Sudman, Sirken, and Cowan 1988). For example, in 
a survey of a rare disease, an initial sample of households 
might be selected at random and data obtained both for 
residents of the households and for their siblings. The 
design-based estimation in these strategies is helped by the 
symmetry of the links and the encompassing of complete 
connected components in the sample, and unbiased esti- 
mators have been obtained for network sampling with many 
different initial designs. 

Another link-tracing procedure for which design-based 
estimators are available is adaptive cluster sampling 
(Thompson 1990, 1997, Thompson and Seber 1996), which 
has been formulated in the graph setting as well as the 
spatial setting. Following selection of an initial sample of 
nodes by any of a number of initial designs, the decision on 
whether to follow links from a node or not depends on the 
value of a variable of interest observed for the node. For 
example, in an epidemiological study of a sexually trans- 
mitted disease, sexual or social links may be followed only 
from respondents who have been infected. Design-unbiased 
estimation methods have been worked out for a wide 
variety of adaptive cluster sampling strategies. 

Design-based methods of inference, such as the design- 
based estimation methods of network sampling, snowball 
sampling, and adaptive cluster sampling, have the advan- 
tage that properties such as design-unbiasedness or consis- 
tency do not depend for their validity on any assumed 
model for the population. On the other hand, these prop- 
erties do depend on the sampling design being carried out 
as specified. The model-based methods described in this 
paper, on the other hand, do depend on an assumed model 
for the population or graph. Their practical advantage is 
that they apply to a wide range of sample selection proce- 
dures, and thus allow more leeway in how the sample is 
actually selected. 

In fact many real studies of hidden and hard-to-reach 
populations use sample selection procedures, including 
link-tracing, that are not readily analyzed based on idealized 
design-induced probabilities. For example, in a study to 
examine the relation of network structure and risk behaviors 


such as needle sharing among drug injectors in the 
Bushwick section of Brooklyn, “index” (initial) respondents 
were used as “auxiliary recruiters” to bring members of 
their networks into the study (Friedman, Neaigus, Jose, 
Curtis, Goldstein, Ildefonso, Rothenberg and Des Jarlais 
1997, Neaigus, Friedman, Goldstein, Idefonseo, Curtis and 
Jose 1995, Neaigus, Friedman, Jose, Goldstein, Curtis, 
Ildefonso and Des Jarlais1996). Only about 61% of the 
linked individuals were actually recruited, however. In a 
long-term study on the heterosexual transmission of HIV 
infection (Rothenberg, Woodhouse, Potterat, Muth, Darrow 
and Klovdahl 1995), the target population of interest 
consisted of commercial sex workers, their paying and 
nonpaying partners, persons who use injectable drugs, and 
the sexual partners of drug users in the Colorado Springs 
area. Persons in the purposively-selected initial sample 
were interviewed and, in addition to their individual charac- 
teristics, identities of their sexual partners were obtained. 
Persons named by two or more respondents were also 
located and interviewed. The wide range of link-tracing 
procedures used in studies such as these has motivated the 
emphasis in this paper on model-based inference methods. 

When we compare the maximum likelihood estimators 
and predictors obtained in this paper with commonly-used 
conventional estimates or data summaries such as sample 
means and proportions of node or link values, we find that in 
most cases the conventional estimates are not the best 
estimates. Similarly, estimators that would be appropriate if 
the data included the whole graph may not be appropriate 
with data on only a sample from the graph. An implication 
of these results is that conventional estimates or unadjusted 
summaries of sample data obtained through link-tracing pro- 
cedures can be misleading if viewed as pertaining to popula- 
tion or whole-graph characteristics. The interpretations of 
this discrepancy provided in this paper give some insight 
into the conditions under which the best estimate would tend 
to be lower, or higher, than the conventional one. 

Notation and basic issues for design and inference in the 
graph setting are presented in section 2. In section 3, a 
wide range of link-tracing procedures, all of which can be 
analyzed using the approach presented in this paper, are 
described. In section 4, a class of graph models that we use 
to illustrate the inference methods of the paper is described. 
Estimative and predictive maximum likelihood methods for 
graph parameters and realized population values are 
described in section 5. 


2. GRAPH MODELS AND SAMPLING 
DESIGNS 


Consider a graph of N nodes (units) labeled 1, 2, ..., NV. 
Associated with the w-th node is a variable of interest Y, . 
We denote the full set of node labels U = {1, 2, ..., M} and 
the sequence of node variables by Y =(Y,,..., ¥y). For 
two distinct nodes wu and v, the indicator variable X,,, equals 
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one if there is an arc (directional link) from u to v and zero 
otherwise. The matrix of arc indicators, having X,, as the 
element in the u-th row and v-th column, is the graph 
adjacency matrix, denoted X. For convenience we will 
assume the diagonal elements X,, are zero. The ordered 
pair (uw, v) is sometimes referred to as a dyad of type 
(Y,,¥,;X,,»X,,)- A graph model is given by a joint 
probability or density f(y, x; v) for outcomes y and x of Y 
and X, respectively, and it may depend on one or more 
unknown parameters w. 

A sample s from the graph is a subset of nodes and a 
subset of node pairs. We can write the combined sample as 
s =(s, 5s), where s“ denotes the subset of nodes se- 
lected for observation of the associated y-values and s 
denotes the subset of node pairs selected for observation of 
the associated x-values. The data consist of the node and 
node-pair labels in the combined sample together with the 
associated node and arc-indicator values, that is d = 

U,(V,W),¥,>X,,: wes”, (vy, wes) or, more simply, 
d=(S,Y,«,X,). Further, itis often convenient to use y, to 
denote the y-values of the nodes in the combined sample 
and x, for the x-values of the node pairs in the combined 
sample, with y. and x. denoting the values of the un- 
sampled nodes and node pairs. Often the sampling proce- 
dure results in a connection between s“ and s™. For 
example, if all relationships from sample nodes to other 
sample nodes, and no others, are recorded, then s = 
sx 5. Tn general, however, the nodes on which y-values 
are recorded and the node pairs on which x-values are 
recorded may be quite unrelated sets. In particular, the link- 
tracing procedures considered in this paper often lead to 
data on links from nodes in s to nodes outside of s“”. 

The sampling design is the procedure by which the 
sample is selected. This selection procedure may be con- 
trolled by the investigators, as is the case with a deliberately 
implemented probability sampling design, or may be be- 
yond the control of the investigators and determined by the 
circumstances of the situation. If the probability of se- 
lecting the sample does not depend on node values y or link 
values x or parameters y involved in the graph model, we 
refer to the design as “conventional.” For a conventional 
design the probability of selecting samples can be written p(s) 
or p(s;@), where @ denotes any unknown parameters 
involved in the design (but not the model), as in a Bernoulli 
sampling with unknown inclusion probability @ for each 
node. The sampling design may depend on one or more 
auxiliary variables that are known for the whole population, 
but that dependence will be left implicit in the notation p(s). 
Conventional designs include the classical probability 
designs such as simple random, systematic, stratified, multi- 
stage, and unequal probability sampling, as well as model- 
based purposive and balanced designs based on auxiliary 
variables. 

If the probability of selecting the sample depends on any 
y or x values, we refer to the design as “adaptive,” since the 
selection procedure adapts to the realized configuration of 


89 


node and link values in the population. In addition, the 
design can involve unknown parameters y. Thus, in 
general the sampling design in the graph setting has a 
selection probability that can be written p(s | y,x; w) 
where y denotes the sequence of node values, x the matrix 
of arc indicator values, and w any parameters involved. 

Likelihood-based inference, such as maximum likeli- 
hood estimation or prediction and Bayes methods, is simpli- 
fied if the design can be ignored at the inference stage. The 
fact that the sampling design does not affect the value of a 
Bayes or likelihood-based estimator in survey sampling was 
noted by Godambe (1966) for designs that do not depend on 
any values of the variable of interest and by Basu (1969) for 
designs that do not depend on values of the variable of 
interest outside the sample. Scott and Smith (1973) showed 
that the design could become relevant to inference when the 
data lacked information about the labels of the units in the 
sample. Rubin (1976) gave exact conditions for a missing 
data mechanism — of which a sampling design can be 
viewed as an example — to be relevant in frequentist and 
likelihood-based inference. For likelihood-based methods 
such as maximum likelihood and Bayes methods, the design 
is “ignorable” if the design or mechanism does not depend 
on values of the variable of interest outside the sample or on 
any parameters in the distribution of those values. For 
frequency-based inference such as design- or model- 
unbiased estimation, however, the design is relevant if it 
depends on any values of the variable of interest, even in 
the sample. Scott (1977) showed that the design is relevant 
to Bayes estimation if auxiliary information used in the 
design is not available at the inference stage. Sugden and 
Smith (1984) gave general and detailed results on when the 
design is relevant in survey sampling situations. Thompson 
and Seber (1996) described adaptive designs in which the 
selection procedure deliberately takes advantage of ob- 
served values of the variable of interest, and discussed the 
relevance of the design in inference from a variety of design 
and model based perspectives. Similar issues of design and 
inference arise with adaptive experimental designs, such as 
medical experiments in which ethical considerations moti- 
vate adaptive treatment allocation to favor the more 
promising treatments as the study progresses (cf. Flournoy 
and Rosenberger 1995, Rosenberger 1996, Wei, Smythe, 
Lin and Park 1990). It is important to underscore that a 
design that is said to be “ignorable” for likelihood-based 
inference might not be ignorable for a frequentist-based 
inference, such as model-unbiased estimation, and that even 
though a design may be ignorable at the inference stage, in 
that for example the way an estimator is calculated does not 
depend on the design used, the design is still relevant a 
priori to the properties of the estimator. 

The sample data d=(s,y,,x,) are a function of the 
sample selected and of the graph values y and x. The 
likelihood can be written 


Liy,d) => p(sly, sw) f(y, y) (1) 
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where the sum is over outcomes (y, x) consistent with the 
data d. Since the y and x values for nodes and node pairs in 
the sample are fixed by the data, the sum is over all possible 
values of the unobserved variables y; and x; and it 
actually represents the marginal probability of the sample 
s selected and the associated observed variables y, and x,. 

Thus, in general the likelihood function depends on both 
the design and the model. The quantity rina f(y, x5), 
based on the model only without consideration of the 
design, was termed the “face-value likelihood” by Dawid 
and Dickey (1977) because inference based on this function 
alone takes the data at face value without considering how 
the data were selected. 

For any design in which the selection of the sample 
depends on graph y and x values only through those values y , 
and x, included in the data, the design probability can be 
moved out of the sum and forms a separate factor in the 
likelihood. If in addition the design and model parameters 
are distinct and not related, the likelihood can be written 


L(g, v,4)=p(s|y,,x39) > flyxuw) (2) 


Ys Xs 


where @ denotes the design parameters and w denotes the 
model parameters. The design then does not affect the value 
of estimators or predictors based on direct likelihood 
methods such as maximum likelihood or Bayes estimators. 
For any such “ignorable” design, the sum in the above 
likelihood, over all values of y and x leading to the given 
data value, is simply the marginal probability of the y and 
x values associated with the sample data. This marginal 
distribution depends on what sample was selected, but does 
not depend on how that sample was selected. For likeli- 
hood-based inference with a design ignorable in this sense, 
the face-value likelihood gives the correct inference. 


3. SOME LINK-TRACING DESIGNS 


A variety of link-tracing designs are described in this 
section. Each of these designs is ignorable in the likelihood 
sense provided the initial sample is selected by an ignorable 
procedure and provided the data include all the values 
involved in the selection procedure. Since for all the 
designs described in this section, the node-pair sample s ‘”) 
has a deterministic functional relationship to the node 
sample s‘), the superscript notation will be omitted and the 
final node sample s“’) will be denoted simply s. 

The simple likelihood methods described in this paper 
apply to a wide range of ignorable link-tracing designs, 
including those described in this section. Further research 
is needed on methods for nonignorable designs, including 
those with nonignorable selection of the initial sample. 
Methods for dealing with nonsampling errors such as non- 
response and reporting errors with link-tracing designs are 
also in need of further development (cf, Thompson 1997). 


3.1 Single-Wave Design 


In a single-wave link-tracing design an initial sample of 
nodes is selected by any ignorable design from the popu- 
lation of nodes in the graph. For each node in the sample, 
nodes adjacent from that node are added to the sample. The 
snowball procedure is assumed to stop after one wave. 
Thus, node v will be added if for some node zw in the initial 
sample x, = 1. 

Let s, denote the set of nodes in the initial sample and s, 
denote the added nodes not in the initial sample. The whole 
sample is s = s,Us,. 

The entire set of labels can be written as the union of three 
disjoint sets, U = s,Us, Us. The values y associated with the 
nodes can be correspondingly ordered as a sequence 
Cy 1, Ys ), where y_, =(y,:u€a@) is the subsequence of y 
restricted to indices in subset ac U. The adjacency matrix x 
is ordered correspondingly and partitioned into submatrices 
X55? Xsps,7 Xsyz and so on, where x, = (x,,:u€ a, Vv € 5). 
Ordering the ‘adjacency matrix in this way facilitates the 
specification of factors in the likelihood. 

With the design above, the probability of selecting 
sample s depends only on x, WU and so can be written 
D(s| Xu)» where Xsy Can also be replaced by its column 
permutation (Xai, Xe Xs ( That is, the probability of 
selecting the final sample s= Sy Us, depends on links from 
the initial sample to other units in the graph, both in s and 
in s. The data consist of (s, y,, Xsu)- Since the design 
does not depend on any x or y values outside the data or on 
model parameter values, the design is ignorable for 
likelihood-based inference. 


3.2 Maulti-Wave Samples 


Consider a snowball sample with k + 1 waves after the 
initial sample. The sample will be denoted s = s,)Us, with 
SoU 5, Uso, U.-.Us,,- An initial sample s,, is selected 
by any design that is ignorable in the likelihood sense. 
Links are followed and every node with an arc from any 
node in s,, and not already in the sample is added to form 
the first-wave sample Soy: That is, be {vx =1 for some 
UES), V€ So) }. Then links are followed in sp, to give the 
second-wave sample S09 = {Vex ek wor ae UES), 
VES) US, } ={v: x,, =1 for some wes USE Vesa, 
Us,,}. Finally, the (ke + 1)-wave sample, denoted simply s,, 
is added by following links from the k-th wave sample so,. 
That is s, ={v: x, =1for some u€ 5), v €5)}. No links 
from s, are followed. 

If So, = = @ for any j < k then sampling stops, so that the 
fiiinber of waves added is less than k if at some point there 
are no links leading out of the current sample to unsampled 
nodes. 

The data consist of sets of node labels in the different 
waves of the sample and the ordered node pairs from s, to 
U, the sequence of node-values y, for all nodes in the 
sample, and the link indicator variables x, ,, from s, to the 
set U of nodes in the graph. Thus the data eorisist of the 
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subgraph data for s,, that is (sy, Ys,° so)? together with 
the node values yi for the nodes in hie ¥inal-wave Spethe 
link indicators x, _ from s, to s,, and the link indicators 
x. ~ from the nodes i in Ss, to the nodés not in the sample. 

"Since the design does not depend on any y or x values 
outside the data nor on any of the graph model parameters, 
the design is ignorable and the structure of the data is 
exactly the same with the (k + 1)-wave snowball as with the 
l-wave snowball design, and with the notation we have 
used the likelihood and estimation formulas are unchanged 
with the more general design. 


3.3 Completed-Wave Designs 


With a completed snowball sample, the procedure of 
adding waves is continued until no further links lead out of 
the sample. Then the number of completed waves K is a 
random variable and sy ,,, = 5, is the first empty set in the 
sequence (Sy), 55> «--)-- The ata are d= (50 Y,, ,Xs,U) OF 
equivalently (sp, Y 597 Xsosq? Xsoz ae Inference can then proceed 
with the same likelthood and estimation formulas but with 
the simplication that the data contains no set s, for which ee 
and X,s, are known but from which links ie unknown. 


> Sy 


3.4 Link-Tracing Adaptive on Node Values 


Consider a design in which the decision to follow the 
links from node u depends on the node value y,. For 
example, in a study on injection drug use, the initial sample 
may contain both users (y, = 1) and nonusers (y, = 0). If 
the investigators choose to follow social links only from 
users, then the design depends adaptively on the node y- 
values as well as the links. Similarly, in a study of sexually 
transmitted diseases, investigators may be instructed to 
follow sexual or social links more frequently from infected 
respondents than from noninfected respondents. The 
design then can be written p(s|y,,x su)» Since the 
selection procedure depends on both node and link values. 
If the data contain all values on which the design depends, 
that 1s,°d =(s, yx * y), then the design is ignorable and 
maximum likelihood inference is simplified as described in 
the following sections. 


3.5 Tracing Only a Subsample of Sample Links 


The designs described above can be generalized to 
procedures in which only a sample of the links leading out 
from node u in s, are followed. Examples include the 
“random walk” design of Klovdahl (1989) and the 
generalization of snowball designs described in Snijders 
(1992). In the random walk design, an initial respondent is 
asked to give the names of several social contacts. One of 
these contacts is chosen at random to be interviewed and 
asked in turn to name several contacts, one of which is 
chosen at random, and so on. In practice, dead ends can 
occur when a respondent either reports no contacts or 
reports only contacts who are already in the sample. In 
such cases investigators either backtrack and try different 
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leads from previous respondents or find a new initial 
respondent. 

With these subsampling link-tracing designs, the 
procedure for selecting the sample, though complicated 
from a design-probability point of view , depends only on 
values in the sample and on links leading from the sample. 
We again assume that the initial sample is obtained by any 
ignorable procedure. Let sy =s Us, Uso, U...Us,, consist 
of all of the waves from which at least some links are 
followed. Thus, s), consists of the nodes not previously 
included obtained by following a subsample of the links 
from nodes in the initial sample s,,, so) consists of the 
nodes not previously included obtained by following a 
subsample of the links from nodes in Soo Dis and so on. 
No links are followed from the final wave s,. Allowing for 
the possibility of ae on node values, the design 
can be written p(s|y,, X,y)» So that with data d= 
GRY. x sol” the design i is ignorable for likelihood-based 
inference. 


3.6 Data from Link-Tracing Designs 


With any of the single or multi-wave link-tracing designs 
described above, it is of considerable practical importance 
what data are recorded. If the data include only the sample 
node labels, the y-values for nodes in the sample, and the 
arc indicators for pairs of units in the sample, that is, 
d =(s,y,,X,,), then the design is nonignorable and must be 
integrated into the likelihood, which can complicate 
analysis. 

Consider also a study in which social links are used in 
the design , to find the sample, but only node characteristics 
(y-values), not relationships are recorded, so that the data 
are d=(s,y,). Then the design is nonignorable. 

If on the other hand the data from the link-tracing design 
include not only the linkages within the sample but the out- 
linkages (or lack thereof) from all but the last wave to the 
rest of the graph, that is, d=(s,y,, Xu)» then the design 
depends only on graph values in the data and so factors out 
of the likelihood. 


4. A GRAPH MODEL WITH LINKS 
RELATED TO NODE VALUES 


The likelihood-based approach described in section 2 
with sample data from link-tracing designs of types des- 
cribed in section 3 will be illustrated using a class of graph 
models described in this section. This class of models 
builds on conditional independence between dyads as in the 
contact models of Frank (1979a) and Wellman, Frank, 
Espinoza, Lundquist and Wilson (1991). Conditional on 
the node values, independence is assumed between dyads, 
with the distribution of links between pairs of nodes 
depending on node value. Thus, unconditionally these 
models have dependence between dyads because of the 
dependence on the node values. In the models of Holland 
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and Leinhardt (1981), dyads are assumed to be independent 
but with distributions that depend on fixed node parameters. 
Wasserman (1980) also assumed independence of dyads in 
modeling the change in a graph over time as a stochastic 
process. Bayesian extensions and stochasstic blockmodels 
of Holland, Laskey, and Leinhardt (1983), Fienberg, Meyer, 
and Wasserman (1985), Wang and Wong (1987), and Frank 
(1988) provide generalizations to joint distributions with 
dependence between node values and graph links. Models 
by Frank and Harary (1982) for randomly colored graphs 
exhibit a similar structure. The Markov graph models of 
Frank and Strauss (1986) provide another approach to 
dependence among dyads but present difficulties for 
maximum likelihood estimation. Review of a variety of 
graph models is found in Wasserman and Faust (1994) and 
Frank (1997). 

The maximum likelihood estimation and prediction 
methods of this paper apply equally to sample data with 
graph models other than the class of stochastic block 
models we have used. With other models, the same condi- 
tions for ignorability apply. We have chosen this class of 
models because it is rich enough to encompass important 
aspects of realism such as dependence between dyads and 
between dyads and node values, and it is simple enough to 
have explicit full-graph maximum likelihood estimators for 
comparison with the estimators based on samples. With 
other classes of models such as the Markov graph models, 
estimation even with full-graph data requires numerical 
methods. 

For practical use of the model based approach it is 
important to have diagnostic tools for evaluations and 
comparisons between alternative models. For example, with 
the two-block model used here the conditional indepen- 
dence of dyads could be tested by counting pairs of dyads 
of different types within and between the blocks. Within 
each block there are three types of dyads and six types of 
pairs of dyads. Between the two blocks there are four types 
of dyads and ten types of pairs of dyads. A Pearson 
goodness-of-fit statistic between observed and expected 
counts of the 22 types of pairs of dyads within and between 
the blocks is asymptotically chi-square distributed with 12 
degress of freedom under the conditional dyad indepen- 
dence assumption. Goodness-of-fit testing for graph models 
is discussed by Holland and Leinhardt (1981) and Frank 
and Strauss (1986), and this direction of research needs 
further development in particular in connection with sample 
data from link-tracing designs. 

In the assumed model the node variables Y ee ke 
independent, identically distributed (i.i.d.) Bernoulli 
random variables with probabilities P(Y, =i) =0,, fori=0, 
1, with 0) +0,=1. Conditional on the node values 
bea Reae) Ge me BAG (X,,>X,,) are independent, for 
Ine oats <N, with conditional distribution given by 
PX Xu) = DY, = 6 Y, =i) = Ay for all combina- 
tions of i = 0, gO. “ile he ‘O. ys and / = 0, 1. For all 
combinations of i and 7, the sums over k and / are denoted 


=V,¥ Ar yw and equal 1. In order to get graph proba- 
sh not depending on node identities, the following sym- 
metry requirements are needed: 1,119 = Ayio1» Ayo1, =» 


Ol? 
Aroio = Aoior A101 = Ao110° Aoo10 = Aooor? 224 Arooo = Apioo: 


The pattern of these restrictions is illustrated in Table 1. 


Table 1 


(x, wv? x, ») 


Wed) (0,0) (0,1) (1,0) (1,1) 


(0,0) @ @e —-® @ 


(0,1) 
(1,0) 


(1,1) © @e__® 6 


With these restrictions, it is convenient to introduce the 
notation 


if (ijkl) = (0110) or (1001), 


otherwise 


/ 
Vis, kel, 
ijkl 
Vij, kel, 


Where Yo + 2Y%1 * Yon = 1, Yio t¥1 + Yn * V2 = 1, and 
Yoo + 22521 + Yx. = 1. We can interpret y,, and y,, as the 
probabilities of dyads with an arc from an unmarked to a 
marked node only and from a marked to an unmarked node 
only, respectively. Moreover, for (ij) # (11), Vy is the pro- 
bability of a dyad with j arcs on i marked and 2 —iun- 
marked nodes. 

It will also be convenient to denote hi i Ly = Oy 
and A,., =B for i= 0, 1 andj = 0, 1. Here a, is the 
probability of an arc from a node of value i to a node of 
value j, and B, is the probability of mutual arcs between k 
marked nodes. 

Let N, denote the total number of nodes with value 7 in 
the graph, for i = 0, 1, so that N, +N, =. Let further 
Mins denote the total number of dyads of type (i&/ ), that is, 
the total year - ordered node pairs (w, v), with u < v, 
such that (Y,, x ik, ) = URL). 

The ieee for the full graph under the model with 
parameters (0, 2) is 


je Te Te 
roxy. -{ te!) | ILA (3) 


In terms of the ys, 


where the Rs are dyad counts corresponding to the pattern 


in Table 1. That is, Ro) =M, DG! OnE 


0000” 01 0010? 
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Ron =Mooivy Rio = Moioo * Miooor Ris = Morar * “Mioro» 
Ri = Moro * Mioor Ria = Morr + Mio Roo = M100 
Ry) = Mio, + Miyior Bop = Mii. Note that Ry, (R,,) is 


the number of dyads with an arc from an unmarked 
(marked) to a marked (unmarked) node only. Also note that 
except for (i) = (11), R, is the number of dyads on i 
marked nodes with / arcs. 

The maximum likelihood estimators with the whole 
graph as data are the proportions 6, = NIN, Y= ROR, 
and 9/, = R/,/.R,, where R, = Nj(N, - 1)/2, R, = N,N, 
and R,=N,(N, - 1)/2. In terms of the As, this means 
Air = Ri / Ry if (ikl) =(0110) or (1001) and Mig = 


/ R,,, otherwise. 


i+j, k+l +) 


5. INFERENCE FROM LINK-TRACING 
DESIGNS 


5.1 Estimating Graph Model Parameters 


Consider any of the link-tracing designs, for which an 
initial or multiwave sample is selected and links out from 
nodes in s, are followed to add the set s, of nodes not in s, 
that are adjacent after nodes in s). The data are d= 
(SY, X,u)» so that the design depends on y and x values 
only through those in the data and is thus ignorable. 

With the graph model described in the previous section, 
the likelihood with the sample data given by equation (2) in 
section 2 is in this case 


N 
L(@, i, da) os P(s | Ys X,u) Dy | I] , (I Soren} 
u=l u<v 


where the sum is over all values y, and x, that are not 
fixed by the sample data. 

Similar to the notation for population counts in the 
previous section, let n,(a) denote the number of nodes 
uea with y, =i for arbitrary subsets acU. Let 
m ijn b) be the count of pairs of nodes (u, v) such that 
ue a, VED, (Vs, Vy>Xyy>%,,) = (kL), and u<v if both w 
and v belong to an b. An index replaced by a dot means 
summation over that index. For instance, according to the 
link-tracing designs described in section 3, only Miike (S554) 
is observed, not m,,,(Sp, 5, ). 

With data from any of the link-tracing designs described 
in section 3, the likelihood function is 


n(s My p)(Sqr So) 
L@aed)=PEsly,x,.)( T18"”)| Taya ; 


ijkl 


ijk ves | (4) 


For the link-tracing designs in which all links, rather than 
a subsample, from the initial sample are traced, all of the 
elements in the submatrix x, - are zero and m9...) = 


n,(S)) for ve Ss, which simplifies the likelihood function to 


m., (Sp, 5 Mipe(Sor V) 
+(e ” Ul ps ey Tae 
J I 
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L(®, dd) = PCs ly, £0) m8” ( mane 


ijkl 
Miike (59,51) n; (So) n(S) 
+| LT Mike Doe Waaee 
ijk i i (5) 


The factor I pai gives the probability of the observed 
node values in the sample. The factor IT eee a gives the 
probability of the observed dyad types within So X So given 
the node values. The factor IT Mie ts gives the probabi- 
lity of the observed dyad types in s, x s,. Since x,,, but not Kin 
is observed, for u€ sy and ve s,, the marginal probability 
that x, =k given y, =i and y, =/ is A,,.. 

The final factor of (5), with square brackets, gives the 
probability that there are no arcs from the initial sample to 
S. For anode v of the n(s ) nodes outside the sample, 0, is 
the probability that y, = 7. From any of the n,(s,) sample 
nodes we€ s,) with y, =i, the conditional probability of no 
link to v, that is, that x, = 0, Miioe: 

More formally, the bracketed term can be obtained by 
conditioning on the number n.(s ) of nodes of type / in S. 
Conditional on nis), the probability that all the link indi- 
cators from s, to § are zero is obtained as follows. From 
the n,(s)) nodes of type iin s, to the nis ) nodes of type 
j in 8, the probability that all links are zero is 4j'.0). 
Using the binomial distribution of n,(s) with the law of 
total probability, the probability that all the links from s, to 
S are zero, given y,, iS 


n(5) n(s ) n.(5) n, (89) n, (8) 
: Q.’ ee 
iin : i . 


j 
ss 9 4 ribo) n(S) p 
: d ut je " (6) 


With the completed-wave design, the above likelihood 
expressions are simplified since the terms Migs (sy, 51) ALE all 
zero, so that the factors involving these terms are all equal 
to one. We also note that A,, = 1 - G,, and Nite =a, can 


be substituted to simplify the likelihood. 


5.1.1 Estimative Likelihood Equations 
The maximum likelihood estimators for the parameters 9, 
aL, and B, are obtained as the common solutions to the 
equations 
dlogL _ dlogl _ dlogl _ 
de da.. dB, 


1 y 


0 (7) 


for i=0, 1, 7=0, 1, k=0, 2. Differentiating the loga- 
rithm of the likelihood (5) with respect to 0, and setting 
equal to zero gives 
dlogL _oOlogl dlogl _ 
d8, 00, 00, 


where the partial derivatives are given by 
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nj (So) 


I, Aiko 
DY OTA iS) 


ijO5« 


AlogL _™') 03) 
OO gay 


foie =O 
Moreover, 


dlogL _ dlogl , dlogl _dlogl  dlogl 
aA j00.0 ongnitnn 


and 


dlog L =. dlogLl , dlogL _OlogL dlogLl 


ap. Fil, Oh RO ee Ch ca mois (9) 


i+j=k 


where the partial derivatives are given by 


dlogL _ MM iny(So» So) Like (So, 51) 
Di Mind Miike 
n,(Sq) -1 
x 
ibn hake 


n,(5>) 
wy, 0, I], Riot: 


It is convenient to write the likelihood equation for 0, as 


n.(s) m6 e)(o — 
ae oS) | n(S)@-1) _9 (10) 
0, 9 9,p + 8% 
where 
n,(Sp) n; (Sp) 
>| 2] ; “|| ‘i 
i=0 Moos i=0 1 - Oi 
Ng(Sq) M4 (So) i 
Note that p=py © Pp; , where p, = (1 -4,,)/(1 - a,9) is 


the ratio between the probabilities of no arc from an i-node 
to a positive and a zero node, respectively. 

An interpretation of the influence of the graph structure 
on estimation of 0, is provided by considering the graph 
parameters a — and hence p — as fixed. Denote the sample 
proportion of positive nodes by 6. =n,(s)/n(s). This is the 
conventional or naive estimator of 0,, using the sample 
proportion of positive nodes. If p = 1, then the maximum 
likelihood estimator 6, would be 6. If p < 1, then the 
maximum likelihood esunuator 6, would be less than 6, 
and if p > 1, 6, Se Ppaticulan O., = 0,5 fora = 0, ‘| 
implies p = 1 and the maximum ketinood estimator is 
6,=6.. 

Consider for instance the case in which for any given 
value of y,, a link from node uw to node v is more likely 
when y, = 1 than when y, =0, so that a,,>a,), fori=0, 1. 
Then 0.) eat, als for i=0, iE so that p < | and 
the Sonera likelihood estimator 6, is less than the con- 
ventional estimator ayy, One could oe that the link-tracing 
design is leading investigators to an unrepresentatively high 


proportion of positive nodes, and the maximum likelihood 
estimator is adjusting for this. 

In specific cases some of the i,,, might be set to zero 
and the likelihood equations have to be modified according- 
ly. Some specific cases will now be illustrated. 


5.1.2 A Symmetric Model 


Symmetric models have i,,, = 0 for k #/ so that arcs 
are always mutual or, equivalently, they can be considered 
as undirected edges. 


The full symmetric model has parameters dy, = Mink for 
Lj eens with 99 + a = |. Here Air = Bij a, = a, 


and 


1,(S9) 
wf t2ta) 
i-0{ 1-8, 


Letting mjy)(Sq5) =T i.) 4.1 WE obtain the maximum 
likelihood estimators as the solutions to the equations 


n(s) m(s) n(s)(1-p) 
Dees aS NTR 6 i 
0, 9 05 + p98, Ud 
S ) 
To Too | N(S ) Np (So) 95 H ve 
Biel Boas Cb DG, aeO 
s )[ On )p9,] 
Ta Tio MG) Uy) % + MS) P 9, -0 (13) 
B, 1 3 1 -B,)(, + p9,) 
on) er) n(S )n, (So) p 8, 
a ; Soe eee Se 14 
B, 1-B, (1 -B,)@, +p) Oat 


If the symmetric model is further simplified by assuming 
By =B, =0, there are only the two parameters 6, and B,, 
and the equations to be solved are 


©) Paetilast V2 Sa) 
and 
N = ny (s)/8, x =B yo), 
N - nj(s)/8, : 


For instance suppose the value y, = 1 indicates injection 
drug use and x, = 1 indicates u and v are injection part- 
ners, so that links are only possible between users and 
tracing these links can only add users to the sample. As an 
illustration, consider a population of size N = 10, 000 with 
statistics 71, (S9)'= 7,\%9 (85) = 43, 7, (s). = 47, and r,, = 42. 
The likelihood equations are 0, B, = 0.0006 and (10000 - 
47/8, )/(10000 - 43/6,) = (1 - By’, leading to the maxi- 
mum Tkelinecd aioe 6, = 0.12 and Bie = 0.005. The 
naive estimator for 9, in this case would be the sample pro- 
portion 47/90 = 0.52 and the naive estimator for B, would 
be 
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42 “f ie = 0.039, 


the proportion of links between users in the sample out of 
the number possible. 


5.1.3 An Asymmetric Model 


A specific asymmetric model has 4,,,, = Niikedier = Miike iter 
so that all arcs are independent. Now f,,, =a, a, and we 
obtain the maximum likelihood estimators as the solutions 


to the equations 


N -n,(s)/0, 
PN —(8)/0, 
and 
Oy My 
1 -a, My. +n; (S,) p/0(N — Ny (s)/8,) 


fori=0,1j=0, 1, where m,, = Mite (S555): 

In particular, if we specify this asymmetric model by 
a, = ija, so that arcs are possible with probability a only 
between marked nodes, then the equations to be solved are 

N . ny (s)/0, as al £3 a)! 0) 
N - no(s)/95 
and 
ah Legh 
M19 + [N®, ~ 1, (8)] 7, So) 


Again, iterative methods are appropriate. 


5.2 Predictive Likelihood for the Total of the 
Unobserved Node Values 


For predicting the value of the unobserved random 
variable n, (5) from the observed data, the relevant likeli- 
hood is 


L[0,4;d,n,(s)] =p(sly, X,u) 


nfs) +n,(S) M ips (Sv Sq) 
* 116; | Aju 
I 


ijkl 


Mp, (Sqr 5) n,(So)n,(5) 
x ( PIT ite | | []Ajo- i : 


ik ij 


n(S) 
N,(S) 


(15) 


Use of the term “prediction” implies only that the object of 
inference is a random variable rather than a fixed, unknown 
parameter, and does not necessarily imply forecasting in 
time. 

The estimative likelihood for 1, (5) is obtained from (15) 
by substituting the estimates 8 and A that maximize the 
(marginal) likelihood (5). The value of n, (5) maximizing 
the estimative likelihood would be the estimative maximum 
likelihood predictor of ,(s). While estimative likelihood 
methods tend to produce reasonable point predictions in 


gS 


many cases, they are less useful as a basis for prediction 
intervals, since the estimates of the parameters are in 
essence treated as the true values (cf, Bjgrnstad 1990, 
1996, Lejeune and Faulkenberry 1982). For this reason, we 
emphasize the use of the profile predictive likelihood. 

Rather than substituting fixed estimators of the para- 
meters into (15) and maximizing this estimative likelihood 
with respect to n,(s), the likelihood (15) is now simulta- 
neously maximized with respect to both parameters and 
n,(S). This means that for each value of n, (S) there are 
parameter values 0,[n,(5)] and Ai lr, (S)] which 
maximize (15) with respect to 6 and 2. Substitution of these 
values into (15) defines the profile likelihood L, (7,6), d] 
for n,(s). The value of n,(S) maximizing the profile 
likelihood is the profile maximum likelihood predictor of 
n,(s). 

For any given value of n,(s), the likelihood is 
maximized where the derivatives with respect to the 
remaining parameters equal zero. The maximizing values 
of 0, are straightforward and are given by 

guetebidantilE (16) 
N 

For the remaining parameters we use d log L/da, and 
d log L/dB, from (8) and (9), with the partial derivatives 
now given by 

0 log L _ TM ijp{Sq>So) , TN ipe(Sq25}) 

On r r 


<a py SOS) 


ikl ijkl ijke ij 0° 


Note that the nj (s) for } =0, 1 are contained in (15) only 


in the factors 
n(s) n(5) 
bi NE 
& (Ss i I J 


where A, = 0, II, Mee Since L is proportional to a binomial 
probability with parameters n(s) and A,/(A,+A,), it 
follows that the maximum of L over n, (5) is obtained for 1, (5) 
equal to the integer closest to 


n(s)A, 
+ 
Ay + AY 


carte a 
2(Ay +A,) 


or either of the integers closest to this number if there are 
two of them. In fact (see, for instance, Feller 1957, p.140), 
the mode of a binomial distribution with parameters (n, p) 
is the integer in the interval [(7 + 1)p-1,(7 + 1)p] or 
either of the endpoints if they are integers. Thus, the mode 
is the integer or the integers that are closest to the interval 
midpoint (n + 1)p - (1/2) =np + (p-q)/2, where q = 1 - p. 

If initial values of the parameter.estimators are obtained 
from the solution of (7) and substituted into the A, then a 
predicted value n, (5) is given as above. If this predicted 
value is inserted into (16) and (17), then new estimates of 
the parameters are obtained that can be substituted into the A_ 
to find a new predicted value of n, (5), continuing until the 
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values converge to the solution minimizing (15). Alterna- 
tively, the solution can be found by direct computation of 
the likelihood (15) for different values of n,(s), substi- 
tuting the solutions obtained from (16) and (17) for the 
parameter values. 


5.2.1 Example: Symmetric Model 


The predictive likelihood equation (15) for the sym- 
metric model is 


nj(s)+n,(5) n(s) 
L[8,B;d,n,(s)] = plsly X40 TTS Nae 


™AV(SQ.8) 4 mM j9 (Sqr5) + (50) 2, (5) 

{16% ieee | (18) 

Let r,, =1,;(So, 8) denote the count of node pairs in sy x s 
with total node value & and total number of links 7. With the 
symmetric model, / can take only the values 0, indicating no 
link between the nodes, or 2, indicating a symmetric link. 
In particular, 755 =7#1g9,;(5o,8), Tig = M%o33)So> 8) + M011 
(Sp, 8),and r,, =1,,,, (So, 8) denote the sample counts of 
links between nodes of total value k, for k = 0, 1, 2, 
respectively. With this notation the last factor in (18) can 
be written 


2 

"ho Ry WAC MAO: 
DB e lee B oy Seat p he os 
k=0 


Denote by c, = c,[n, (S)] the number of possible node 
pairs in s, x U having total value &, so that 


Ce It > n,(S9)n,(S) 


iJ 
itj=k 


= DF nj(so)n(s) +, )]- 

ij 

i+j=k 
For any given value of n, (Ss), the likelihood is maximized 
by ae =[ (sn (@VN for i = 0, 1 and B, = Ty, PO, ter 
k = 0, N42 Note that 6 and the B, are functions of the 
unobserved variable 7, (S). 

The profile predienve likelihood function for 7, (Ss) is 

obtained by substituting the maximizing values 6 and B, 
for the parameters in (18), giving 


n,(s) +n, (5) n,(s) +n,(S) 
N 


n(s) hes 2 ee 
“ln Ne Snaps 


which is a function of n,(S) alone. The maximum profile 
likelihood predictor of 1, (5), easily obtained by straight- 
forward computation, is an integer between 0 and n(s) 
giving the largest value of (19). 


L{n, (5); d] =p(s nyo I | 


5.3 On Assessing Accuracy of Estimates 


For confidence intervals and other forms of inference, 
the inverse of the observed Fisher information I(@) is sug- 
gested, where @ is the vector of parameter maximum likeli- 
hood estimates and [is the matrix of negated second deriva- 
tives of the log likelihood function evaluated at those esti- 
mated values. The use of the observed, as opposed to 
expected, Fisher information to assess the accuracy of an 
estimate is described in Efron and Hinkley (1978). More 
recently, Lindsay and Li (1997) argue that the observed 
information gives a better assessment of the realized, as 
opposed to expected, error of the estimate. In developing 
large-sample approximations to the properties of the 
estimators of 6 and A it is important to make appropriate 
assumptions about how A depends on JN so that the graph 
model and the sample do not degenerate. See for instance 
the asymptotic results for some simple graph models given 
by Palmer (1985). 

As with the calculation of the maximum likelihood 
estimates themselves, the calculation of the observed 
information matrix is not affected by the link-tracing samp- 
ling design, since the design is ignorable for likelihood 
based on inference. This is in contrast to the expected 
Fisher information, the value of which is affected by the 
design in addition to the graph model, unless the design is 
a conventional one not depending on any y or x values. 

For a (1 - €)-level prediction interval for a random vari- 
able such as n, (5), one method would be to use a central 
region having mass (1 - €) of the normalized profile likeli- 
hood function for n, (Ss) (cf, Bjgrnstad 1990, 1996). For 
the symmetric model, the (1 - €) prediction interval for 
n,(S), is readily obtained by computing (19) for 
n, (5) =0,1,2,..., until the computed values become negli- 
gible, porcaltaine by dividing by the cumulative total 
ye o£, and using the ¢/2 and 1 - e/2 quantiles as the 
interval endpoints. 
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Calibration and Restricted Weights 


ALAIN THEBERGE! 


ABSTRACT 


To better understand the impact of imposing a restricted region on calibration weights, the author reviews the latter’s 
asymptotic behaviour. Necessary and sufficient conditions are provided for the existence of a solution to the calibration 
equation with weights within given intervals. A more general formulation of the calibration problem leads to a compromise 
between the need to satisfy the calibration equation and the attempt to obtain weights that are close to Horvitz-Thompson 
weights. If the requirements for the calibration equation are relaxed, then various estimation methods with restricted weights 
can be used. The estimators that are introduced usually have the same asymptotic properties as the calibration estimator with 
no weight restrictions, and some have weights which can be calculated explicitly, without any iterative process. The author 
shows how these estimators can be adapted to take advantage of a synthetic estimator. An approach similar to that used to 


restrict weights is applied to outliers. 


KEY WORDS: Small domains; Moore-Penrose inverse; Inequality solutions; Asymptotic properties; Outliers. 


1. INTRODUCTION 


The calibration estimator has good asymptotic prop- 
erties. However, for samples of small size, or if calibration 
is done at the domain level and some of the domains 
involve few observations, the weights of such an estimator 
can include extreme values. One way of overcoming this 
problem consists in using the calibration method with dis- 
tance measurements which restrict the weights of observa- 
tions to certain intervals about the sampling weights. This 
approach was developed by Deville and Sarndal (1992). 
Other methods aimed at providing robust estimates satis- 
fying the calibration equation can be found in Duchesne 
(1999). That paper contains an extensive bibliography on 
robust estimators. However, there is no guaranteed solution 
to the calibration equation with restricted weights. Even 
when such weights exist, the statistician might prefer 
solving the problem of extreme weights by relaxing some- 
what the requirements for the calibration equation, instead 
of tightening the constraints on the weights by using a dis- 
tance measurement that is more “restrictive”. This paper 
provides a formulation of the calibration problem which 
offers more flexibility to the statistician. The problem in 
fact is one of minimization similar to that encountered in 
ridge regression. Bardsley and Chambers (1984) encoun- 
tered this same minimization problem in their search for 
model-based estimators. This formulation of the calibration 
problem can be used to restrict weights without the use of 
special distances between calibrated weights and Horvitz- 
Thompson weights. Rao and Singh (1997) combined this 
approach with iterative methods using distance measure- 
ments. Other ways of restricting weights will also be 
reviewed. 

In the next section, the calibration method is outlined 
without applying limits to the values of weights. The 


calibration problem thus outlined does not assume there is 
a solution to the calibration equation. The asymptotic prop- 
erties of calibrated weights are discussed. These properties 
have a bearing on the asymptotic behaviour of the esti- 
mators whose weights are restricted. In section 3, necessary 
and sufficient conditions are provided for the existence of 
restricted weights which satisfy the calibration equation. 
Section 4 discusses how the estimation problem can be 
formulated by varying the importance attributed to the cali- 
bration equation. Section 5 provides various means of 
restricting weights without recourse to a specific distance. 
Section 6 introduces an estimator with restricted weights 
which is useful for small domains. Finally, in section 7, out- 
liers are discussed in terms of a method similar to that used 
to deal with extreme weights. 


2. CALIBRATION 


Let Ye R*”? denote a matrix of d variables of interest 
for a population of size N, and let c € R% denote a vector of 
known constants; a sample s of size n is drawn, and the sub- 
script s is used to designate the sub-vectors or sub-matrices 
corresponding to the sample. We wish to estimate Y'c 
using Y'w,, where w.€ R” is a weight vector for the 
sampled units. For a vector v and a positive diagonal matrix 
F of identical dimension, we define || v||, = v'Fv. For an 
auxiliary information matrix XeR”?, AeR™ the dia- 
gonal matrix of sampling weights, given positive diagonal 
matrices U.eR"™” and TeR?*, we seek, among the 
weight vectors w,¢R” which minimize || X/w, - x ‘ely, 
the one which minimizes D,(w,) = || w, -A,c, | U,: This 
formulation of the problem, which does not assume the 
existence of weights satisfying the calibration equation, 
X)w, =X ‘ec, can be found in Théberge (1999). The 
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solution represents the vector of calibrated weights w_.,. 
We have 


Wea =A, A UGX Te nex UX 
TNX CAMA a) 


where F' denotes the Moore-Penrose inverse of the matrix 
F. 

To better review the asymptotic properties of calibration 
estimators with restricted weights, let us now examine the 
behaviour of w.,, when 1 > c. We assume there exists an 
asymptotic setup in which the size of the population and the 
size of the sample tend towards infinity (see for example 
Isaki and Fuller (1982)), and for which we have 


Y'c = O(N") (y 2 0) 


KC AAC, 2059 oN 7) (2) 


Pexcuian = O, (n). 


It follows that (T!2x! U. axe Te\is O,(n- 1) | since one 
of i properties of the Moore-Penrose i inverse of a matrix 
Fis F'FF' = F‘. Usually, we can expect to have 
y= 1 ails each element of the vector c has a value of 1 
(estimate of a total), and y = 0 when each element of c has 
a value of 1/N (estimate of a mean). For conditions (2) we 
therefore have, 


WiC XE ad x de X oh yi 


Tua Xlee XA €.) 


i] 


O,(n ")O,(n eNOS 


O,(n PN"). (3) 


Thus w,,, —A,c, converges in probability to 0, if 


Lim n 32N7 = 0. 
n, N+ 


For an asymptotic setup such as that of Brewer (1979) in 
which the sampling fraction n/N is constant, or any setup 
for which the sampling fraction converges to a positive 
number, this condition is verified if a — Oe. 
Writing w,,, =A,c,+U,'X.T'? HT!” (X'c-X'Ac.) 
where H, = T nx" U, Dee 1 we have 
DW.) =(X'¢ - X'A.c,)'T HH, 
T'?(X'c -X/Ac.) 
=(X’e -X'Ac,)'T!H)T? (X'e-X'Ac.) 
=, “12 ary -1 -12 
O,(n ND) O; GreeiOs (nage Ni") 


OS SEN AY) (4) 


Théberge: Calibration and Restricted Weights 


Again for an asymptotic setup in which the sampling 
fraction converges to a positive number, we have D.(w..,) 
converging in probability to 0, if y<1. Thus there are 
cases, e.g. for the estimate of a total, where w,,, - Ac, 
fae Soe in probability to 0, but where D,(w,,,) = 
| Woot Ot U, does not converge to 0. 


3. CALIBRATION EQUATION SOLUTIONS 
AND RESTRICTED WEIGHTS 


Even in the absence of weight restrictions, there might 
not be a solution to the calibration equation. By applying 
Graybill (1983, 113) to the calibration problem, we find that 
the calibration equation X'w, = X‘c can be solved if and 
only if (X, x: \'X'e = xe c. If there is a solution, the 
calibrated weights might be negative or exceptionally large. 
Deville and Sarndal (1992) proposed using various distance 
measures other than a weighted sum of squares to measure 
the distance between Horvitz-Thompson weights and cali- 
brated weights, so as to restrict the weights to certain inter- 
vals while satisfying the calibration equation. This approach 
can only work if there are in these intervals weights which 
satisfy the calibration equation. The main goal of this 
section is to find necessary and sufficient conditions for the 
existence of a weight vector w, within given bounds, such 
that the estimates of totals for auxiliary variables are also 
bounded. In other words, we are seeking conditions for the 
existence of a vector w, such that w’)<w,<w” and 
t) < XK! wo st, where wl), wH) 1) and t are given. 
In particular, by assuming ft) = t) = X’c, we would ob- 
tain conditions for the existence of weights restricted to the 
intervals w)<w <w), satisfying the calibration 
equation. 

A first step is provided by the following Fan (1956) 
theorem. It is formulated here for a matrix M of finite 
dimension, although the proof provided by Fan also applies 
to a matrix of infinite dimension. The theorem uses the 
kernel of M’, N(M'), defined as the set of vectors a such 
that M’a =0. 


Theorem: Let Me R”*” and /—E R”, Swe R” such that 
Mw >I ifandonlyifforany 2 >0 in N(M’), we have 
<0. 

Corollary: Let Me R”*” and J,he R”, J we R” such that 
l<Mw<h if and only if first 7<h and secondly 
hE N(M') = -l'X_<h'd,, where 2, = max(d,0) and 
X_ =min(A, 0) with the extrema taken elementwise. 

The corollary is obtained by using the theorem with 


w()-(4) m>-(2) 


Let p denote the dimension of V(M"). If p is equal to 
zero, then 4 € N(M’) implies 4 = 0, and the condition of 
the theorem (or the similar condition of the corollary) is 
obviously met. If p is equal to one, then 7A€¢ N(M) implies 
that 2 is a multiple of a vector z, and it is sufficient to 
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check the condition for 2 =z and 4 = -z. If we use the 
property (-A)_ = -(A)_,, the problem outlined at the begin- 
ning of the section can now be resolved if X, is a vector. 
The corollary with 


I, we) wt) 
ie ' i). 4 Gale , ( at 


and the fact that 


spans N(M’), provide the necessary and sufficient 
conditions 


wh) < wh 
t)) < 4D 6) 
CX) ws X ae Bee) 
(L) wy (A) 1 yw) 
CA NW A) oe. 


The third inequality in (5) states that the weighted total of 
the auxiliary variable must not be greater than ¢“, when 
the smallest possible weight w“) is given to units for 
which the auxiliary variable is positive, and when the 
greatest possible weight w“” is given to units for which 
the auxiliary variable is negative. The fourth inequality in 
(5) states that the weighted total of the auxiliary variable 
must not be less than t‘/), when the largest possible weight 
is given to units for which the auxiliary variable is positive, 
and when the smallest possible weight is given to units for 
which the auxiliary variable is negative. 

Even for p> 1, it is sufficient to check the condition of 
the corollary for a finite number of values of 2. Let 
VeR”*?, 2 < p<m denote a matrix whose columns form 
a basis for N(M’). It is always possible to construct V 
such that p of its rows, v,,¥,, ..., v,,. are the unit vectors of 
R?, and we will assume that V is of this form. It will be 
shown in Appendix A that it is sufficient to check the 
condition of the corollary for vectors 4 = V@ and 4 = -V@q, 
where @ = (Q),..., 9, )’ is a non-zero vector satisfying 
vig =O for a subset of (p-1) linearly independent 


vectors v,. We must therefore check the condition at the 
mostfor | ™ | vectors @, i.e. at the most 2 m \ values 
of 2. pal p-l 

Using the corollary with 


ii. we) wt) 
Mis i). ; | Mofo A veil 


form a basis for N(M’), we obtain the following necessary 
and sufficient conditions for the existence of a solution to 
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the problem mentioned at the beginning of this section 
whenever X,€R””? with p>1. We must have w‘) < w#) | 
i) Dana foreach. subset “of (p-1) linearly 
independent rows of 

L, 


(X,9). w ~ 9/2 


it is necessary that 


IA 


(AO). Weert a” 


(6) 
“(X,9) Ww +911 


lA 


(X,9) Ww? — gt 


for a non-zero vector @ € R? orthogonal to each row of the 
subset. The second inequality in (6) is obtained from the 
first by changing the sign of . 

If V.,,€ R?”? is a non-singular matrix whose rows are 
rows of We then each column of Ve is a vector perpendi- 
cular to (p - 1) linearly independent rows of V. Hence the 
following result: 


There exists a weight vector w, such that w‘” < w, < w) 
and t”) <X/w.<t if and only if w < 0, t{)) < 
t) and 


aly, Sy Sy, 
(x, Vas) wi) th (Vii) Ae Ss = @ Va wi) 


WV 


sub 


yr 


2 2 : (7) 
(XV) WO + Va) t® < (KV) we 


2 era t() 


for all non-singular matrixes V., € R’*? whose rows are 
sub 


4 | P 
. 


These conditions are somewhat redundant. For example, 
if inequalities (7) are met for V,,,, = V,, then they are neces- 
sarily met for any matrix V, obtained from V, through a 
permutation of rows. 

Another example is provided by weighting observations 
in a contingency table. Assuming N. = iW; 
(P= 2229 R; pS yds © )eerhere Hi asethe number of 
observations in cell (i, " of a contingency table and w,, iS 
the weight of each of these observations, we wish to know 
if there are weights w,, i such that N, ij Satisfies certain con- 
straints. For example, motivated by the problem of conver- 
gence of the raking ratio procedure, Bacharach (1965) pro- 
vided necessary and sufficient conditions for the existence 
ea such that Ny =O iN, =N, G=1,....0), 
oe iN N, (i=1,..., R), where the values of N, and N, 
are given. The following result, demonstrated in 


Appendix B, is more general. 
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: ‘ 
For arbitrary constants Ny ) Nie Nn‘ ) nit), 


vn, me nw, and N wy there are N,, such that 


Rave haaG 


; BG? 
(8) 


> ( nO - my] (9) 
J i 


J 
(mi? wP) aw” 


ieS ele 


I 
DEEN eis (wr oe wr) 


i€S 


< [me apes Ny) Ne 
ieS 


JET 


forany S etl 2". 2 RYETAL 2 cy 
The number of inequalities to be checked can be 
reduced. For example, instead of checking 


Danes lt) dae? Dd a 
jJeT i¢S ieS JET 


forvanyao clos, fo, and als... Cy, “itteansbe 
readily shown that an equivalent procedure would be to 
check that 
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forany 7 ail, 2,.3Co. 


4. MITIGATED CALIBRATION 


There may be dissatisfaction with the two-step approach 
of calibration, where an attempt is first made to find weight 
vectors that best satisfy the calibration equation, and then 
from this set of vectors to find the one which comes closest 
to Horvitz-Thompson weights. For small samples, this 
method may lead to weights which the statistician will find 
too far from Horvitz-Thompson weights. 

There may be a preference for varying the importance 
attributed to the calibration equation relative to the norm of 
w,-A,c,. Thus, there may be a desire to find a weight 
vector w, which minimizes 


MMe 2 Acc, 
XJ, -X'c 


U, 0 


Z 


> 


where 


Va= 


0 aT 


and a > 0. We then minimize 
| w,-A,¢, Ig, +a X/w, - X’elp = 
D,(w,) +a XJw, - X'e I; : 

A similar minimization problem is encountered with ridge 
regression. For a = 0 the solution is provided by Horvitz- 
Thompson weights w, = A,c,. For a>0, we seek w,(a) 
minimizing | K(w, - A,c,) - bh bs where K= 
U,,,.X,)', b=(0,,.,,,(X’e-XA,c,)')' and 0,,,, ER” isa 
row vector of zeros. Ben-Israel and Greville (1980) yields 


w.(a) - Ac, = (K'VK)'K'Vb. (10) 


Thus by substituting the values of K, V, and 5 we obtain 
W.(0)) =4,c/4 a( 0; 


+aX TX!) 1X T(X'e = XA). (1) 
It is easily shown that 
a(UiraX 1x’) UXT 
[UPR T Lex Ure 
hence 
w(a)=A.e,+UX(a'T! 
POY Mea en 12) 


| 


| 
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becomes Y’c + 


where Y = X B,(a) and 


The estimator Y,w,(a) thus 


(¥, - ¥,)'A,c 


s&s? 


Boje (Xu, X40 5) I xu. 
The vector of regression coefficients, then, is the one ob- 
tained with ridge regression. Just as the calibration method, 
and the generalized regression method described by 
Sarndal, Swensson and Wretman (1992), lead to the same 
estimators, a similar parallel can be drawn between miti- 
gated calibration and ridge regression. 

On the basis of equation (12), we can also use Ben-Israel 
and Greville (1980), and the fact that Ft = F’(FF')' with 
F = T'?X'U,"'”, to show that 


lim w,(a) = 
a+ © 


cal * 


This result was to be expected, since finding the, vector 
w,(a) which minimizes D,(w,) +a|_X/w,-X’' el; when 
a@- is equivalent to finding the weight vector which 
minimizes D.(w,) among those which minimize 
| X/w, - Xe |p. 

Rao and Singh (1997) defined tolerances for each of the 
Pp constraints of the calibration equation, and they estab- 
lished a relationship between these tolerances and the 
matrix aT. 

For a € [0, ~[ the function w.(a) is represented by a 
curve in R” which links point A,c, to point w,,.. If p=1, 
i.e. if X is a vector, this curve is a line segment. In fact, in 
this case the matrix (a 'T~! +.X/U, 'X.)“! and the vector 
X'c - X{A.c, are scalars, and the weights w,(a) given by 
(12) are therefore equal to Horvitz-Thompson weights plus 
a multiple of vector UAIX® And again for p = 1, we have 


Tm Wal) = = c. 


yee 
#[(X'e=K'Ac)/(X'U,'X,)1U,'X, 
which leads to the estimator 
AW, PAYA 
+ ((¥! U,'X,)/ (X1U,'X,)(X'e - X/A.c,) 
Taking U = A ‘diag(X), we obtain the ratio estimator 
VAcos 


Vdd N(x A 


nxl Ss” nx 


NCTC XA CS), 


where 1,.., € e R?”° is a matrix of ones. 

Ben- Israel and Greville (1980, 111, exercise 15) showed 
that D.(w,(a)) is an increasing monotonic function of a. 
Note however that for a unit kes, | w,(a) -a,¢,| is not 
necessarily a monotonic function of a. As a increases, the 
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weight vector w.(a) moves away from the Horvitz- 
Thompson weight vector, but this does not necessarily 
apply to each coordinate of the vector. 

In this article, mitigated calibration is used to restrict 
weights, i.e. when the size of the sample is relatively small. 
It can easily be shown, however, that for an asymptotic 
setup satisfying (2) and for which B, (a) -B(a) - O in 
probability, with 


B(a) = (X'U'X+0'T')'X'U"Y, 


we have ¥’w (a.) is an asymptotically unbiased estimator 
whose asymptotic variance is 


(Y- Y")' diag(c)(AIIA - 1,,,,)diag(c)(¥Y- Y*), 


where Y* = X B(a.),II is the matrix of inclusion probabili- 
ties of order 2, and diag(c) is the diagonal matrix formed 
from vector c. 


5. ESTIMATION METHODS WITH 
RESTRICTED WEIGHTS 


In order to avoid obtaining weights having extreme 
values, we may wish to force the weight vector to be within 
a given region. This restricted region will be assumed to be 
convex and closed, and A,c, will be assumed to be a point 
in this region. For example, if WL) <A.c.<w™, we may 
wish to restrict the weights to region R, ={w,:w< 
w.<w)}. We will assume that 


lim wW-A.c. <0 and lim w” 


n- © n+ 


-Ac, > 0. 


The approach described in section 3 consists in selecting a 
distance measure between calibrated weights and Horvitz- 
Thompson weights which will provide weights that satisfy 
the calibration equation and which lie in the restricted 
region, should such weights in fact exist. The approach 
dealt with in this section is to temperate the requirement 
that the calibration equation be satisfied when the vector of 
calibration weights w.,, is outside the restricted region. 
Various means to temperate this requirement lead to 
different weighting methods. 

When w,,, lies outside the restricted region, we could for 
example look for those points on the curve w,(a) para- 
metered by a > 0 which are on the border of this region. 
One property of these points is that they solve the minimiza- 
tion problem described in section 4 for corresponding 
values of a, thus through matrix 7, the importance of each 
calibration equation can be weighted. Using the example of 
the restricted region provided above, if 


w.., = lim w.(a) 
ee 


can be used as a 
=w,.(a.) with 


lies within this region, then w_..) = Wa) 


restricted weight vector, otherwise w,,., 
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a <> can be chosen such that w,(a) is on the boundary of 
the restricted region. If the asymptotic setup is such that 
conditions (2) are met with y < 3/2 then for n sufficiently 
large, the probability that w,,, will be within the restricted 
region is equal to one. In fact, we have w,,, -A,c, 
converging in probability to 0. The asymptotic properties 
of the estimator using the restricted weights, w_..,, are 
therefore identical to those of the calibration estimator. It is 
worth noting that since |w,-a,c,| is not necessarily a 
monotonic function of a, it is possible for w,(a) to be on 
the boundary of the restricted region for several values of 
a, even if the restricted region is convex. Finding all these 
values is not necessarily a simple matter, and a decision has 
to be taken as to which value to use. 

Another option for restricting weights would be to use as 
a restricted region those weights w, which satisfy 
D,(w,) < / for a bound / > 0. Then w,,,, = w,,, is taken as 
a restricted weight vector if w,,, lies in the restricted region, 
otherwise we seek a@>0O such that D,(w,(a)) =/. This 
value of a is unique and can be found through iteration. 
Next we calculate the weights w,,.. = w,(a) which corres- 
pond to this value of a using equation (12). If the 
asymptotic setup is such that conditions (2) are met with 
y <1, and if / does not vary with n, then for n sufficiently 
large, the probability that w.,, will be within the restricted 
region is equal to one. In fact, we have D,(w,,,) conver- 
ging in probability to 0. The asymptotic properties of the 
estimator using restricted weights, w,,. ,, are then identical 
to those of the calibration estimator. Unfortunately, when 
estimating a total, we must expect to have y = 1. In order to 
overcome this snag, we can use lyn as a bound, instead of 
/. We can justify this bound on the basis that the length of 
the main diagonal of a hypercube of R” is equal to the 
diameter of the sphere which circumscribes this hypercube, 
whereas the diameter of the sphere inscribed in this same 
hypercube is smaller by a factor of yn. The fact remains 
that a statistician might be uncomfortable using an asympto- 
tic setup where the bound increases with the size of the 
sample. Furthermore, with this approach, the weights of the 
observations cannot be limited individually. Only the dis- 
tance between the restricted weight vector and the Horvitz- 
Thompson weight vector is controlled. 

With the methods described above, we look for those 
points on curve w,(a) which lie on the boundary of the 
restricted region. The value of a for which w,(a.) lies on 
the boundary of the restricted region must often be found 
iteratively. It would be simpler to replace the curve w, (a) 
by the line segment linking A,c, to w,,,. For the restricted 
region R,, the restricted weight vector would be 
W.<3 =. if w,,, i8 in the restricted region, otherwise 
W_.,3 would be equal to the point at which the line segment 
crosses the boundary of restricted region, i.e. 


Wres 3 = A.c, i E( Weal r A.c,) 2 
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where 


a= a {max [(w\ - A,c,)/ (Ww. -A,¢,); 


(Ww - Ac.)/(w,, -A,¢,)]}, 


vector division being elementwise, the maximum of the two 
vectors being elementwise, and min providing the mini- 
mum element. We could also consider the weight vector of 
the restricted region, w,,.,, which comes closest to Waal 
Again for restricted region R,,, we would have 


w w)), wt) ), 


es4 = min [max (w 


cal? 
The asymptotic properties of estimators using restricted 
weights w.., or w_,., are identical to those of the calibra- 
tion estimator, as long as w.,, - A,c, converges in probabi- 
lity to 0, which is usually the case. 

One interesting property of all the approaches discussed 
in this section is that, no matter what the restricted region, 
the existence of restricted weights is guaranteed. This is not 
always the case when using an approach based on distance 
measures. A simple example will now be introduced to 
allow comparisons between a few approaches. 

We wish to estimate a total on the basis of a simple 
random sample of size 2 in a population of size 20. In other 
words, c =1,),, and a = 10(1,5,,). We use the auxiliary 
information vector X = (1, 2,3,...,20)’, assume that the 
selected sample is s = {2,12} and choose U as a diagonal 
matrix with u,, = x, = k. A rectangular restricted region 
is provided using points w~ =(0,0)’ and w“ =(20, 13)’. 
In other words, the weight of the first sample unit must be 
greater than O and less than 20, whereas the weight of the 
second sample unit must be greater than 0 and less than 13. 

Under these conditions, the calibrated weights 
w.,,= (15, 5)’ lie outside the restricted region. Since p = 1, 
weights w.(a) lie on the line segment which links 
A.c, = (10, 10)’ to w,,,. We therefore have w,.., =w...3, 
which means that the two methods give the same result. In 
this case, we have w..., =Ww_.., = (13, 13)’. The method 
which consists in choosing that point in the restricted region 
which lies closest to the calibrated weights yields 
W <4 ~ (15, 13)’. On the other hand, if we look for pee the 
restricted weights obtained while requiring that the calibra- 
tion equation be satisfied and while using a distance mea- 
surement which assumes an infinite value outside the 
restricted region, then there is no solution. In fact, for any 
weight in the restricted region X/w,< 196, whereas 
X'c=210. If we had, say, w“” = (30, 13)’, then using 
D,(w,) as a distance measurement within the restricted 
region we would have w___. = (27, 13)’. These weights are 
fairly distant from  w,,,= (15,15)’ and _ from 
A,c, =(10, 10)’. Such is the price to be paid for insisting on 
having weights which meet the calibration equation. 
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6. ESTIMATORS FOR DOMAINS WITH 
A SYNTHETIC COMPONENT 


Restricted weights are used because of the properties of 
the calibration estimator for small sample sizes. For large 
sample sizes, we normally have w.., - A,c, converging in 
probability to 0, i.e. weights that are not problematic. A 
statistician faced with a problem of extreme weights must 
therefore in all likelihood also face another problem of 
small sample sizes, i.e. estimation for small domains. This 
section introduces an estimator whose asymptotic properties 
are those of the calibration estimator, but which uses 
restricted weights and takes advantage of a synthetic 
estimator. 

Let Y=X B, denote a synthetic estimate for Y, we have 


Y'w, = (XB,)'w, 


I 


B he W, 


ll 


R 


B/x Ka 


(XB,)'c 


a 


Y'c (13) 


with equality at the third step if the weights satisfy the 
calibration equation X/w, = X’'c. The weights w_., given 
by (1) minimize |X! WX” clr. We can theretone 
estimate Y’c using 


= (Y,-Y,)'w. + Y'c. (14) 


There will be equality between this estimator and 
estimator Y’w_., once the sample is sufficiently large for 
the calibration equation to be satisfied and for w,,, to lie in 
the restricted region, i.e. once w.,. = w.,,. The asymptotic 
properties of these two estimators are therefore identical 
under certain conditions discussed in the previous section. 
The advantage of using estimator t is that it provides a 


synthetic estimate when columns of Y, and Y, are zero. 


7. OUTLIERS 


Outliers could be dealt with in much the same way as 
extreme weights. The strategy is the following: we adopt a 
restricted region for Y/w,,,, we show that when n is 
sufficiently large Y/ w.,, lies within the restricted region, 
and we adopt a more “reasonable” estimator to replace 
Y’w.,, in those cases where Y/w,,, lies outside the 
restricted region. For a stratified sample, we would 


normally have one restricted region per stratum. 
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In section 2, it was shown that under certain conditions 
for the asymptotic setup, w.,, - A.c, = =O. 2 oN) We 
thus have Y/w.,, - ViAc. = 0, ce IRN) and if we 
assume that 


FAR CoO5 (in HENNE (15) 


then Y'w,,,- Y'¢ = O,(n “!?N). An expert (or a group of 
experts) could determine on the basis of information 
gathered independently of survey data that it would not be 
reasonable to have Y/w,,, outside a certain region. If Y’c 
lies within the restricted region (i.e. if the expert does not 
find it unreasonable to have an estimate of the parameter 
which would be equal to the true value, Y’c, of the 
parameter), if y = 0, and if the restricted region does not 
vary with n or N (or if y = 1, and the restricted region varies 
in proportion to N), then for sufficiently large n, the proba- 
bility that Y’w.., will lie within the restricted region is 


cal 
equal to one. In those cases where Y’ w 


ca Les outside the 
restricted region, we could use as an estimate the point in 
the restricted region that lies closest to Yw.,, or we could 
assume that the weight of the few observations that are 
deemed outliers is equal to one, and distribute their original 
weights (less the number of outliers) among the observa- 
tions that are not outliers. The asymptotic properties of this 
modified estimator used to deal with outliers are then 
identical to those of the unmodified estimator. 

In the case of a non-stratified sample, this method is 
relatively easy to apply. If however the sample is stratified, 
and if constraints are imposed on estimates for each 
stratum, then we have two additional problems. First, if the 
asymptotic setup is such that the number of strata increases 
in proportion to the size of the sample, then the assumption 
given in (15) does not hold, since the mean sample size per 
stratum remains constant as n>. We need to determine 
whether it is reasonable to adopt an asymptotic setup in 
which the number of strata is constant (or increases less 
rapidly than n). Such an asymptotic setup is less plausible 
if the number of observations per stratum is small. The 
second problem is linked to the difficulty for the expert to 
impose constraints on estimates for each of the strata. The 
greater the number of strata, the greater the risk that Y’c 
will not lie in the restricted region defined by the expert. In 
fact, in the case of a stratified sample, it is preferable for the 
expert to use information that is independent of the survey 
data, in order to ensure strata homogeneity, prior to fina- 
lizing stratification. In other words, it is preferable to use 
the information available before the survey, in order to 
prevent outliers, rather than to correct them. If the informa- 
tion has been used in such a way that, before the survey, 
there is no reason to believe that there is any unrepresen- 
tative observation in any stratum, then there is no justifi- 
cation for assuming the opposite once the data have been 
collected. 
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8. CONCLUSION 


If for large sample sizes the calibrated weights remain 
within a restricted region, then the asymptotic properties of 
the estimator with restricted weights are obviously identical 
to those of the calibration estimator. For a given asymptotic 
setup, we can usually expect to have w,,, — A,c,converging 
in probability to 0, i.e. for sufficiently large sample sizes the 
calibrated weights w.,, will remain within the restricted 
region R,, if Ac, lies within R,,. However, we have seen 
that for the estimate of a total, we do not necessarily have 
convergence to 0 for D,(w,,,). We must therefore avoid 
having a restricted region defined by || w,-A,c ¢, ly, < Tat 
least if we are estimating a total and not a mean. 

We have provided necessary and sufficient conditions 
for the existence of weights restricted to intervals which 
satisfy the calibration equation. If such weights do not exist, 
the idea of satisfying the calibration equation exactly must 
be abandoned. The problem of calibration with restricted 
weights can be reformulated in such a way that a solution 
will always be possible. Some of the approaches described 
in this paper make it possible to obtain a solution without 
recourse to iterative methods. These are simple methods 
that are easy to interpret. The asymptotic properties of these 
estimators are usually identical to those of the calibration 
estimator without weight restrictions. 

The problem of extreme weights is encountered for small 
sample sizes, thus the problem of estimating for small 
domains should be considered simultaneously. It is possible 
to take advantage of synthetic estimators while using an 
estimator with restricted weights having good asymptotic 
properties. 

It is also possible to modify the calibration estimator, or 
any other asymptotically consistent estimator, so as to deal 
‘with outliers. The conditions under which this modified 
estimator will have asymptotic properties identical to those 
of the unmodified estimator are not easily verified, just as 
it is difficult to verify whether an outlier is in fact unrepre- 
sentative. However, such conditions make it possible to 
identify those factors which allow an estimator that is 
corrected for outliers to be statistically valid. 
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APPENDIX A 


We wish to verify that Q(@) =1'(V@)_ - h'(Vo), has 
a value of zero or less. First, it is easily shown that this is 
true for a vector @, if and only if it is true for a vector kp 
with arbitrary k>0. Only the direction of @ matters. It is 
therefore sufficient to verify the condition for @ of norm 
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equal to one. For this proof, we will use the l,-norm of 9, 
lol, = }”_, |@,|. Vectors @ with |@| Vs 4 are located 
in hyperplanes whose intersections lie on points orthogonal 
to the unit vectors, i.e. points at least one of whose 
coordinates is zero. Function Q varies linearly except at 
points @ orthogonal to one or more rows of V. Even when 
the domain of Q is restricted to vectors @ with ||| Te =] 
that are orthogonal to 0 < j<(p-1) linearly independent 
rows of V, function Q still varies linearly except at points 
orthogonal to other rows of V or orthogonal to unit vectors 
(which are likewise rows of V). The maximum of Q for 
||| a = 1 is therefore reached at a point @ orthogonal to 
(p -1) linearly independent rows of V. It is therefore 
sufficient to verify the condition for two vectors of opposite 
direction which are orthogonal to (p-1)_ linearly 
independent rows of V, and this for each subset of (p - 1) 
linearly independent rows of V. 


APPENDIX B 


Let vec(F) denote the vector obtained by piling 
successive columns of matrix F ¢ R°*? with the first 
column on top, and let the Kronecker product of two 
matrices F and G be defined as 

IiO SiG 
FeG = 


Sn E nN, Hee, 


The result is derived from the corollary in section 3 with 


(B1) 


Tec 
M = ee , w = vec((N,)’), 
1, p2T : 
Ls rc 


vec((N,\)") 


i Ny 
(B2) 
ee ye 
[p= he Die 
Ape! Nick 
Ne Neg 
n” Noe 
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Only a finite set of conditions need be verified, first by 
noting that the columns of 


Te @ Vey, Wry @Le Trex 
vee Tr, Onxc Dex a 
OcxR I Dox1 
Dir Dix I 


form a basis for N(M’). In other words, M’'V =0, the 
columns of V are linearly independent, and V(M’) is of 
dimension R+C+1. Note also that the last R+C+1 rows of V 
are the unit vectors. Finally, we verify the conditions of the 
corollary for all vectors 4 = V@ and 4 = -V@q, where @ is 
orthogonal to R+C linearly independent rows of V. This 
last step is described in greater detail in the following 
paragraph. 

An arbitrary subset of R+C linearly independent rows of 
V which includes the last row of V is denoted L, and the 
subset of all rows of V which are linear combinations of 
rows of L is denoted L*. If L* includes row RC+i (i= 1, ..., 
R) if and only if i¢ Sc ({1,2,...,R}, and includes row 
RC+R+j (j=1,..., C) if and only if j ¢ Tc {1,2, ..., C}, then 
we set @= (9, -@,,0)', where the i-th element of 
QM, € R* is equal to 1 if i € S and to 0 otherwise, and the 
j-th element of @, € R© is equal to 1 if 7 € T and to 0 
otherwise. Then 


Vo we ((-@, ® 1.) ¥ | @Q,)', Qs; ak Fes 


therefore @ is orthogonal to all rows of LZ”, and all the more 
so @ is orthogonal to all rows of ZL. Likewise, vector 
@* =(@,@,, -1)’ is orthogonal to all rows of a subset of 
R+C linearly independent rows of V which includes row 
RC+i (i=1,..., R) if and only if i ¢ S, and includes row 
RC+R+j (= 1, ..., C) if and only if 7 ¢€ 7, but does not 
include the last row of V. The condition -1'A_ < #'2, with 
X4=Vo provides the fifth set of inequalities in (9). 
Likewise, by assuming 4 equal to -V@, V@* and -Vq* 
we obtain the last three sets of inequalities in (9). 
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A Cautionary Note on Adjusting Weights for Nonresponse 


WILLARD C. LOSINGER, LINDSEY P. GARBER, BRUCE A. WAGNER and GEORGE W. HILL! 


ABSTRACT 


For surveys which involve more than one stage of data collection, one method recommended for adjusting weights for 
nonresponse (after the first stage of data collection) entails utilizing auxiliary variables (from previous stages of data 
collection) which are identified as predictors of nonresponse. In the final stage of data collection for the United States 
National Animal Health Monitoring System’s Beef ’97 Study, two variables were identified that clearly separated eligible 
producers by their propensity to respond. However, these variables were noticeably inferior to simple region by herd-size 
categories as predictors of responses that eligible producers gave for other questions in previous data-collection stages. 
Therefore, we decided to form weight-adjustment classes by region and herd size, even though other variables were greater 
predictors of response. When selecting auxiliary variables to adjust weights for nonresponse, we recommend that survey 
Statisticians also evaluate the extent to which these auxiliary variables are related to data which nonrespondents would have 
provided. Using auxiliary variables which exhibit the greatest variation in response propensity may result in the greatest 
variation in weight-adjustment factors, but may bias population estimates for parameters unrelated to the chosen auxiliary 


variables. 


KEY WORDS: Nonresponse bias; Response propensity; Logistic regression; National survey. 


1. INTRODUCTION 


In multistage surveys where some participants fail to 
respond during the final stage of data collection, one has 
considerable information about final-stage nonrespondents 
from previous stages of the survey. Rizzo, Kalton and Brick 
(1996) presented several methods for selecting auxiliary 
variables and adjusting weights for nonresponse when a 
large number of characteristics of the nonrespondents were 
known. These methods concentrated on identifying and 
using characteristics that discriminated between respon- 
dents and eligible nonrespondents. However, by adjusting 
weights based on specific variables which demonstrate the 
greatest difference in response rates, one may potentially 
introduce bias in the survey estimates if these variables are 
unrelated to responses that would have been given by 
nonrespondents during the final stage of data collection. 
Therefore, one should also utilize data from the previous 
stages of data collection to determine whether the chosen 
auxiliary variables are linked to other characteristics of 
those eligible to participate in the survey. 

The Beef ’97 Study (of the National Animal Health 
Monitoring System (NAHMS) of the United States 
Department of Agriculture (USDA)) took place in 23 states 
and involved three stages of data collection. In the first 
stage (December 30, 1996 through February 3, 1997), enu- 
merators from the USDA: National Agricultural Statistics 
Service collected data on general management practices 
from 2,713 agricultural operations with one or more beef 
cows. First-stage respondents who had five or more beef 
cows on January 1, 1997 were eligible to continue in the 


second stage of data collection (from March 3 through 
May 23, 1997), provided they had at least one beef cow and 
remained in business at the time of the second stage of data 
collection. A total of 1,190 producers participated in the 
second stage of data collection, which involved an on-farm 
visit by a veterinary medical officer or animal health tech- 
nician and concentrated on the health management of the 
beef cattle. 

All operations that participated in the second stage of 
data collection were eligible to participate in the third and 
final stage of data collection (August 1, 1997 through 
January 31, 1998). A total of 952 (80.0%) eligible opera- 
tions responded in the final stage. From the first two stages 
of data collection, a considerable amount of information 
was available on the 238 nonrespondents for the final stage 
of data collection. The purpose of this note is to describe 
the methods that were evaluated for adjusting the sample 
weights for nonresponse in the final stage of data collection 
for the NAHMS Beef ’97 Study. 

In addition to region and herd-size (based on the number 
of beef cows) categories, 45 variables based on data col- 
lected during the first two stages of interviews were eva- 
luated for their impact on final-stage response rates. A 
stepwise variable selection procedure, with region and herd 
size forced into a logistic regression model and a signifi- 
cance level of 0.05 for other variables to enter and remain 
in the model, was used (Table 1). The logistic regression 
analysis demonstrated that there were some differences in 
final-stage response by region, but that differences in 
response by herd size were not significant. Increased 
nonresponse was associated with having only one breeding 


PRC: Losinger, L.P. Garber, B.A. Wagner and G.W. Hill, United States Department of Agriculture, Animal and Plant Health Inspection Service, Veterinary 
Services, Center for Epidemiology and Animal Health, 555 South Howes Street, Suite 200, Fort Collins, Colorado 80521 U.S.A. 
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season and not consulting a veterinarian to treat or diagnose 
disease during 1996. The potential use of the logistic- 
regression variables as auxiliary variables in creating cells 
to adjust weights for final-stage nonresponse was examined. 
Four categorization schemes for nonresponse weight 
adjustment were proposed: 


1. The traditional region by herd size scheme with 15 
cells. 


2. Region by herd size except in the West, which was 
subdivided by the number of breeding seasons, for a 
grand total of 14 cells. 


3. Subdividing the cells of option 2 (by either of the 
auxiliary variables) if the difference in response rate 
(between the two new subdivisions) was at least ten 
percent and at least 20 respondents remained in each 
cell. Two subdivisions occurred, which yielded a total 
of 16 cells. 


4. Continuing the subdivision of categories, based on the 
greatest difference in response rate, until a minimum 
number of respondents (no fewer than 20) remained in 
each cell. This yielded a total of 24 cells. 


Table 1 
Results of Stepwise Logistic Regression to Identify Variables 
Associated With Nonresponse to the Final Stage of Data 
Collection for the National Animal Health Monitoring System’s 
Beef ‘97 Study. Based on 1,190 Eligible Operations and 238 


Nonrespondents 

Variable/ Parameter Pp 

Response Estimate 

Intercept 0.369 0.181 

Region 
Northcentral 0.851 0.000 
Southcentral 0.822 0.000 
Central 2.062 0.000 
Southest 1.164 0.000 
West 1.000 

Number of beef cows 
1-49 0.299 0.106 
50 - 99 0.146 0.151 
100 + 1.000 

Number of breeding Seasons 
1 -.370 0.039 
>I1 or no set season 1.000 

A veterinarian was consulted to 

treat or diagnose disease in 1996 
Yes 0.441 0.005 
No 1.000 


Adjustment factors for weights of final-stage respon- 
dents were computed by dividing the sum of second-stage 
weights for eligible operations by the sum of second-stage 
weights for final-stage respondents within each cell. 

Since the establishment of cells for schemes 2 through 4 
was based on variables which demonstrated the greatest 
differences in response rates, differences in adjustment 
factors increased for particular subcategories from scheme 
1 to scheme 4. For example, for the first scheme, adjustment 
factors for the Western region were 1.897, 1.504 and 1.579 
for the small, medium and large herd size categories 
respectively. For the second scheme, adjustment factors in 
the Western region were 1.334 for operations that did not 
have one defined breeding season, and 1.875 for operations 
that did have one defined breeding season. For the third 
scheme, operations in the West that had one defined 
breeding season were split into two cells based on whether 
they had used a veterinarian to diagnose or treat disease 
during 1996: operations that had indicated “‘yes” received 
a weight adjustment of 1.548, while operations that had 
indicated “no” received a weight adjustment of 2.326. 

To investigate how well the proposed auxiliary variables 
might have related to overall management strategies, we 
selected additional variables from the first two stages of 
data collection, and, within each region, examined differ- 
ences in these variables by herd size category, number of 
breeding seasons, and whether a veterinarian had been 
consulted to diagnose or treat disease during 1996. Table 2 
presents some representative results for the Western region. 
Some herd-size differences existed in the percent of opera- 
tions that had one set breeding season and the percent of 
operations that had consulted a veterinarian during 1996. 
However, the percent of operations that had consulted a 
veterinarian was practically identical for operations that had 
one set breeding season versus operations that did not have 
one set breeding season, and vice versa. In addition, the 
percent of operations that vaccinated heifers for brucellosis 
and the percent of operations that implanted calves with a 
growth promotant exhibited a wider range by herd size 
category than by the other two proposed auxiliary variables. 
Moreover, mean weaning age and mean calf death loss 
varied more by herd size than by either number of breeding 
seasons or by whether a veterinarian was consulted. Similar 
patterns were noticed for other regions. 

Although herd size was not a Statistically significant 
predictor of participation in the final stage of data collection 
for the NAHMS Beef ’97 Study (table 1), herd size was 
found to be more highly related to a number of question- 
naire variables than either of the additional proposed auxi- 
liary variables which derived from the logistic regression 
analysis. Therefore, we utilized the tradional region by herd 
size category scheme to perform the nonresponse weight 
adjustment for the final stage of data collection for the 
NAHMS Beef 97 Study. 
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Table 2 
For 261 Western-Region Operations Eligible to Participate in the 
Third and Final Phase of Data Collection for the United States 
National Animal Monitoring System’s 1997 Beef ’97 Study 
(August 1 through January 31, 1998), Responses to Selected 
Variables From the First two Phases of Data Collection by 
Auxiliary Variables Examined for Weight Adjustment for the 
Final Stage of Data Collection 


Variables selected from the first two 
stages of data collection 
Auxiliary variables 
proposed for weight 
adjustment for third-stage 


nonresponse 
Percent Mean 
Number of beef cows 
1-49 COVE S0SarOsal es ora 215) 16:3 
50 - 99 69.2 59.6 80.8 26.9 232 3.9 
100+ 88.2 70.1 85.4 52.8 223 4.1 
Number of breeding 
seasons 
1 = nO251 69.81 7,0) 9223 5.1 
>I1 or no set season OS Oeoloen45.0 225. 64.5 
A veterinarian was 
consulted to treat or 
diagnose disease in 1996 
Yes I925 =)" 69:8) 28:1 222 435 
No 80.0 -— 842 442 223 4.6 


Variables selected from the first two phases of data collection: 


1 = Operations with one set breeding season 

2 = Operations that consulted a veterinarian to treat or diagnose 
disease in 1996 

3 = Operations that vaccinate any heifers for brucellosis 

4 = Operations that implanted any calves with a growth promotant 
prior to or at weaning during 1996 

5 = Average age (in days) of calves at weaning 

6 = Percent of calves that died in 1996 


Researchers using survey data depend on sample weights 
to produce population parameter estimates that are approxi- 
mately unbiased. In the final stage of data collection for the 
NAHMS Beef ’97 Study, a logistic regression analysis 
identified two variables that were superior to herd size as 
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predictors of nonresponse in the final stage of data collec- 
tion. However, these variables were generally inferior to 
herd size in differentiating how producers responded to a 
number of key questions related to operation management. 
Using these two variables to establish categories for weight 
adjustment for nonresponse could have reduced bias in 
estimates of parameters (from the third stage of data 
collection) with which they were correlated. However, 
estimates of parameters not correlated with these variables 
could have been distorted. Therefore, we chose the tradi- 
tional approach of performing the nonresponse weight 
adjustment by region and herd size categories. 

Identifying variables that are good predictors of panel 
nonresponse is a good practice in any multistage survey. 
Prior to using these variables to adjust weights for unit 
nonresponse, we recommend that survey statisticians first 
follow some procedures to determine the extent to which 
these variables are linked to other characteristics of those 
eligible to complete the survey. Adjusting the weights based 
solely on variables that prove to be good predictors of panel 
nonresponse could potentially result in warped population 
estimates if these variables are not also good predictors of 
data that nonrespondents would have provided on the 
survey instrument. 
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Applications to Survey Sampling 


JULIET POPPER SHAFFER! 


ABSTRACT 


Survey statisticians frequently use superpopulation linear regression models. The Gauss-Markov theorem, assuming fixed 
regressors or conditioning on observed values of regressors, asserts that the standard estimators of regression coefficients 
are best linear unbiased. Shaffer (1991) showed that the Gauss-Markov theorem doesn’t apply when the regressors are 
random if some aspects of the population distribution of the regressors are known, and introduced an alternative estimator 
with better properties than the standard estimator under some conditions. This paper derives some generalizations, and 
notes an optimality property (locally best linear unbiasedness) of the generalized alternative estimator. Implications for 


estimation in surveys are noted. 


KEY WORDS: Regression analysis; Gauss-Markov theorem; Survey sampling; Unbiased estimation; Optimality; Best 


linear unbiased estimation. 


1. INTRODUCTION 


In the standard linear regression model for a sample of 
observations, 


Y=XB+e, (1) 


the matrix of regressors, X, is assumed to be a known, 
fixed matrix. Shaffer (1991) showed that when X is 
assumed to be random, the Gauss-Markov theorem does not 
hold in general, and described an alternative estimator that 
is more accurate when f is close to zero. Shaffer gave two 
applications of her results, to estimates of B and associated 
population quantities in multivariate normal superpopula- 
tion models and to ratio estimation of population means and 
totals. 

In the present paper, three generalizations of these results 
are derived. 


(a) The results are generalized from a model in which the 
sample covariance matrix of the errors € is o7J, 
where I is the nxn identity matrix, to the case in 
which the covariance matrix ) of € is o°B, where B 
is a known, fixed positive-definite matrix, and to some 
situations in which B is random (since it 1s the cova- 
riance matrix of a randomly-selected sample of 
regressor values). 


(b) A generalized estimator is derived that performs well 
when the coefficient vector B is close to any pre-speci- 
fied coefficient vector Bp. 


(c) Acondition is given for design-unbiasedness of esti- 
mators of population means and totals based on the 
generalized estimator of B. 


Some results under the general model (1) will be given 
first. Then, modifications that apply to the sample survey 
situation will be discussed. 

Under Model (1) with Y =07J, the Gauss-Markov 
theorem asserts that the sample estimator 


B = (X'X) 1X’Y, (2) 


is a best linear unbiased estimator (BLUE) if X is regarded 
as a fixed matrix. If the rows of X are treated as realizations 
of random vectors x,,i=1,...,”, the Gauss-Markov 
theorem can be interpreted as an assertion that the estimator 
in (2) has minimum variance in the class of estimators 
linear in Y and conditionally unbiased, given these realized 
values of X. However, the use of the term “unbiased”  with- 
out qualification generally means unconditional unbiased- 
ness. If the requirement of unbiasedness is interpreted to 
mean unbiased unconditionally, i.e., on the average over 
random vectors with values in X, Shaffer (1991) showed 
that the Gauss-Markov theorem doesn’t apply when 
E(X'X) is known. In that case, the conditionally biased 
estimator 


B* =(E(X’X) 1X’) (3) 


is unconditionally unbiased and has smaller variance than p 
when f is small. In fact, when E(X’X) is known, no 
BLUE exists. 

Comparison of the variances of (2) and (3) under various 
modeling assumptions, aside from the implications for esti- 
mating the coefficients themselves, gives insight into the 
conditions under which various estimators of other para- 
meters of the populations have desirable properties, both 
model-based and design-based. 


' Juliet Popper Shaffer, University of California, Department of Statistics, 367 Evans Hall, #3860, Berkeley, CA 94720-3860, U.S.A. 


E-mail: shaffer@berkeley.edu. 
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2. GENERALIZATION OF THE COVARIANCE 
MATRIX OF ¢€ 


If the covariance matrix of € is of the form o?B, where 
B is a known, fixed positive-definite matrix, the Gauss- 
Markov theorem applies to the generalized estimator 


B =-[x'B XxX] x'B'Y. (4) 


The proofs in Shaffer (1991) generalize directly to show 
that, if E(X’B 4X) is known, the estimator 


B* = [E(X’BX)| | X’BIY (5) 


has smaller variance than (4) when f is sufficiently close to 
zero. The (unconditional) variances of (4) and (5) are 


Dy = EX’ Bx) Jo? (6) 
and 


Yy = [EX BX] 0? 
(7) 
+ Var{[E(X'B1X)]} "(xB Lx) p}. 


When B = 0, Shaffer shows that (7) is smaller than (6), and 
therefore, assuming continuity of (7) as a function of B, it is 
smaller than (6) when B is in a neighborhood of zero. 

The results will now be applied in the sample survey 
context. Let X,, refer to the Nx p matrix, and Y,, to the 
Nx 1 vector, in a finite population. If the N population 
elements are considered to be a sample from an infinite 
hypothetical population of potential elements satisfying (1), 
and if a sample of size n of the finite population is taken, 
the proofs in Shaffer (1991) generalize directly to show that 


B= [E One ie » iia (8) 
and 


p; = [EC xiB,'x,)] XEBUY: (9) 


have variances smaller than those of their corresponding 
conditional versions By and B, respectively, if B is close to 
zero, where the expectation in (8) is over the infinite popu- 
lation of hypothetical elements, and the expectation in (9) 
is over either the same infinite population or over the finite 
population of N elements satisfying (1). In order to apply 
these results, the expectations in (8) and (9) have to be 
known. 

If X,, is to be regarded as fixed, the population model 
can be written as 


Yy =XyB + Ey; (10) 
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where €, is a vector of randomly distributed error terms as 
in (1). Under Model (10), B,, and fx are identical, but B,, 
is still distinct from B,. Under Model (10), for a random 
sample of size n, if 


E[(X; B,'X,)/n| = (Xj By Xy)/N, (11) 
the alternative estimator can be written in the form 
B; =|(n/N)(X, By Xy)]| X1B,'Y,. (12) 


In model (10), Equations (11) and (12) will apply if B,, 
is diagonal and the sampling plan is self-weighting, and 
under some other conditions and sampling plans, e.g., if B,, 
is block (cluster) diagonal and complete clusters are 
sampled. If B,, is diagonal, B,, is not necessarily fixed. For 
example, suppose a population consists of both men and 
women, and the variances of the two sexes on the charac- 
teristic of interest are known and are different. In that case, 
if a self-weighting sample is taken, and Model (10) is 
assumed to hold in both subpopulations, B, will be 
diagonal, with entries that are a function of the proportions 
of the two genders in the sample. 


3. LOCALLY BEST LINEAR UNBIASED 
ESTIMATION 


Under the model (1), the estimator (5) is the locally best 
linear unbiased estimator (LBLUE) when 8 = 0; i.e., the 
estimator, linear in Y and unbiased for B with smallest 
variance in a neighborhood of B =0. Furthermore, the 
generalized linear estimator 


Byp,) = By *[ECX’B'X)]"[X’B(Y-XB,)], (13) 


allowing for the addition of a constant, is the LBLUE at 
B = B), for an arbitrary vector B,. The proof of these results 
in given in Appendix A. This generalized estimator (13) 
could be useful in a survey sampling situation in which it 
was reasonably sure that B would be close to some specified 
value. The variance of (13) is easily shown to equal (7) 
with (B - B,) substituted for B. (See Appendix A.) Under 
Model (10) estimators (8), (9), and (12) generalize to 


Bip») =Bo *[ECXy By Xw)] [Xu Bu Fy -Xy Bo), 14) 


Bip,» =Bo */ECK,B,'X,)] [XiB,'(,-X,B,)], (15) 
and 


Bis, ) =By +[(n/N)Xy By Xy)| |XiB;'(¥,-X,Bp )) (16) 


respectively. 
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4. CONDITIONS FOR DESIGN 
UNBIASEDNESS 


Assume the Model (10) holds, and that the uncondition- 
ally unbiased estimator can be expressed in the form (16). 
Suppose there exists a px 1—vector g, such that By 34 wee 
1,, and, for every sample of size n, B,, De ag =, AS i 
and 1, are vectors of ones of length. N and n, respectively. 
Then, “given a simple random sample, 


(a) the estimator 


Ky Boe, ay 


is a design-unbiased estimator of Y,,where X, = 


(1/N)1,,X,,,and 


Ys.) Ss 


(b) re , is a generalized difference estimator of Yi. 


The proof is given in Appendix B. 


Note that a vector g satisfying the conditions of this 
theorem exists if the model includes an intercept (i.e., Xj) 
includes a column of ones) or if B,, is diagonal and the 
variance is proportional to the values of one of the regres- 
sors. Many applications of regression modeling to sample 
survey estimation are based on models that incorporate 
these assumptions. Sarndal, Swensson and Wretman (1991, 
p. 231 and 232) discuss these and more general models, and 
Chapter, 6, section 4 of that reference has examples of 
commonly applied models incorporating these assumptions. 
Chapter 6 as a whole discusses both the general difference 
estimator of NV Y, and the analogous general regression 
estimator based on B,. The material in that Chapter also 
suggests generalizations of these results to more complex 
estimators and sampling plans. 


5. DISCUSSION 


To apply the results to estimates of properties of a finite 
population, it will be assumed that the matrix B is diagonal 
or has the special block-diagonal form and associated 
sampling plan discussed above. From the results in section 
3, it follows that the estimator (17) of Y,, has smaller 
variance than the estimator 


¥g = XB, (18) 


when f is close to B. Note that (18) can be written 


EES peese x: hk (19) 
N iés i¢s 


and X’ is the i-th row of X, and S is the set of elements in 
the sample. Royall (1970) showed that the best linear 
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model-unbiased estimator of (unbiased conditionally on 
the obtained sample) is 


we ¥+>ox'p 6, (20) 
les i€s 

In some important cases, the first term in (20) is equal to the 
first term in (19), in which case (20) and (19) are identical. 
This will be true, for example, if B = oJ and the model 
(10) contains an intercept, or if p = 1 and B is diagonal 
with diagonal entries proportional to the values of the single 
regressor. In such cases, (20) and (19) are identical, and 
the design-unbiased and unconditionally-model-unbiased 
estimator (17) has a smaller expected squared discrepancy 
from Y, yn than the best linear conditionally-model-unbiased 
estimator (20) when B is close to B,. Furthermore, if the 
sampling fraction is negligible, (17) has smaller expected 
squared discrepancy than (20) when f is close to B,, even 
without the requirement that the first terms of (20) and (19) 
be equal. 

If p is replaced by p* in (20), the resulting estimator is 
no longer unconditionally unbiased. It can be shown, how- 
ever, using concepts of dependence (Lehmann, 1966) that 
under the conditions on B noted at the beginning of this 
section, the resulting estimator will have smaller expected 
squared discrepancy from ¥ than (20) and (19) even with- 
out the further restrictions noted in the previous paragraph. 


6. CONCLUSION 


Since the conditions under which the estimator (5) of B 
is more efficient than the estimator (4) are very restrictive, 
and the estimators of population characteristics based on (5) 
can be derived in other ways, the results given here may be 
of more theoretical than practical interest. The results do 
give additional insight into some situations in which simple 
estimators like the sample mean and the generalized differ- 
ence estimator are more efficient in estimating the popula- 
tion mean than are ratio estimators, poststratified estima- 
tors, regression estimators and other complex estimators. 
The equations (6) and (7) for comparative variances of (4) 
and (5) provide an alternative method of comparing respec- 
tive variances under different regression models and differ- 
ent values of B. Many of these results hold under very 
simple sampling plans, but it should be possible to general- 
ize them to more complex, unequal probability sampling 
plans. 
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APPENDIX A 


Proof that B, is LBLUE at B, 


Assume model (1), with Var( YIX) = o’B. (The general 
proof given here applies directly to the model (10) as well.) 
Consider the sample estimator 


Bigy =By +[E(X'B*X)] 'X’B(Y-XB,). 


Let t= B -B, and Z=Y- XB). Then E(Z1X) = Xt, 
Var (ZIX)=0°B, and 7° =[E(X'B'X)|' X'B'Z = 
Bis ~ Bo 

Thus, the properties of Bis) at B = B, are the same as 
those of p* = Be (0) at. B = 0, so ‘without loss of generality it 
will be shown that Bo is LBLUE at B =0. Also without 
loss of generality, it will be assumed that B = I. 

Let C’(X)Y be an arbitrary unconditionally-unbiased 
estimator of B, where C(X) is a matrix of functions of X, of 
the same dimensions as X. The requirement of uncondi- 
tional unbiasedness necessitates the restriction E[C’(X)X] = 
(Shaffer 1991). Conditioning first on X and then using the 
expression for unconditional variance, the variance of 
C’(X)Y is ELC'(X)C(X)]o° + Var(C’(X) XB). Since we 
are considering variance at B =0, only the first term is 
nonzero. Letting C’(X) = [E(X’x fr 1X’, the variance of p* 
is [E(X’X) |'0? 

Let B be an arbitrary unconditionally-unbiased estimator 
of the form C(x) ¥. Then Var (B) = Var (p* 0) + 
Var (B - Bp Jit 2Cov (f*, B - p* ); sO Var(p* )s Var(B) if 
Cov(p*, B - B* ) 2 0, or if Cov(p’, B) > Var(B*). An easy 
calculation, using the restriction EC’ (X)X l= I, shows that 
Cov (p’, B) = Var (p* ), which proves that Bos, ) is LBLUE 


at Bo. 


APPENDIX B 


Proof of the Result in Section 4 
X, Bigg ~KABo* UN) 1yXql QIN NX, By'Xy)"| 
XB, (¥, -X,Bo) 
= XjBy + (1/n)g'X, By Xy(XyBy Xp) 
XB, (Y, ~ X,Bo) 
= Xi Bo + A/a)lcY,, - X,By) 


= X\Bo + ¥,, ~ X;,Bo- (B.1) 
where B,, and B, are the appropriate population and 
sample matrices, respectively. The final expression in (B.1) 
is the generalized difference estimator based on a value B, 
chosen independently of the sample. This proves part (b) of 
the result; since the difference estimator is unbiased for Y 
in a self-weighting sample, the result in (a) follows. 
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GUIDELINES FOR MANUSCRIPTS 


Before having a manuscript typed for submission, please examine a recent issue of Survey Methodology (Vol. 19, No. 1 and 
onward) as a guide and note particularly the points below. Accepted articles must be submitted in machine-readable form 
preferably in WordPerfect. Other word processors are acceptable, but these also require paper copies for formulas and figures. 


4.1 


4.2 
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D2. 


Layout 


Manuscripts should be typed on white bond paper of standard size (8% x 11 inch), one side only, entirely double spaced 
with margins of at least 1% inches on all sides. 

The manuscripts should be divided into numbered sections with suitable verbal titles. 

The name and address of each author should be given as a footnote on the first page of the manuscript. 
Acknowledgements should appear at the end of the text. 

Any appendix should be placed after the acknowledgements but before the list of references. 


Abstract 


The manuscript should begin with an abstract consisting of one paragraph followed by three to six key words. Avoid 
mathematical expressions in the abstract. 


Style 


Avoid footnotes, abbreviations, and acronyms. 

Mathematical symbols will be italicized unless specified otherwise except for functional symbols such as “exp(-)” and 
“log(-)”, etc. 

Short formulae should be left in the text but everything in the text should fit in single spacing. Long and important equations 
should be separated from the text and numbered consecutively with arabic numerals on the right if they are to be referred 
to later. 

Write fractions in the text using a solidus. 

Distinguish between ambiguous characters, (e.g., w, @; 0, O, 0; 1, 1). 

Italics are used for emphasis. Indicate italics by underlining on the manuscript. 


Figures and Tables 


All figures and tables should be numbered consecutively with arabic numerals, with titles which are as nearly self 
explanatory as possible, at the bottom for figures and at the top for tables. 

They should be put on separate pages with an indication of their appropriate placement in the text. (Normally they should 
appear near where they are first referred to). 


References 


References in the text should be cited with authors’ names and the date of publication. If part of a reference is cited, indicate 
after the reference, e.g., Cochran (1977, p. 164). 

The list of references at the end of the manuscript should be arranged alphabetically and for the same author chronologically. 
Distinguish publications of the same author in the same year by attaching a, b, c to the year of publication. Journal titles 
should not be abbreviated. Follow the same format used in recent issues. 
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LESLIE KISH 
(1910 - 2000) 


This issue is dedicated to the memory of Leslie Kish. His 
infectious joie de vivre, his deep concern for the oppressed and 
the underprivileged, and his profound contributions to survey 
methodology and statistics have been, and continue to be, an 
inspiration to so many. 
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In This Issue 


This issue is dedicated to Leslie Kish, who passed away this fall at the age of 90. It is remarkable 
to note that to the end of his life Professor Kish continued to propose and develop new ideas in 
statistics and survey methodology, as evidenced by his article “Cumulating/Combining Population 
Surveys” which appeared one year ago in the 25" anniversary issue of this journal. This issue of 
Survey Methodology opens with a reflection on his life and contributions to statistics written by Ivan 
Fellegi. 

The paper by Haines, Pollock and Pantula examines the estimates of a total when two incomplete 
list frames are combined with an area frame. The authors give suggestions on appropriate population 
totals to account for the incompleteness of the frames. In addition, their models allow for the fact 
that larger sampling units are more likely to be included on the incomplete list frames. 

Beaumont proposes an estimation method which reduces the bias induced by a response 
mechanism that depends on the variable of interest, known as a nonignorable response mechanism. 
The proposed method requires one model for the variable of interest and one model for the response 
probability. The method is considered robust with respect to the hypothesis of normality since it is 
constructed in such a way that there is no need to specify the error distribution of model involving 
the variable of interest, unlike the method of maximum likelihood. The author also proposes a simple 
method of verifying the validity of the hypothesis of error normality whenever nonresponse is not 
ignorable. 

Spencer considers the problem of estimating the design effect due to weighting when the selection 
probabilities are correlated with the variable of interest. Using a regression representation of the 
population, Spencer presents an approximation to the design effect when the selection probabilities 
are correlated with the variable of interest. 

Biemer and Bushery use the Markov assumption on labour force transitions to identify classifi- 
cation errors in labour force data. Using this methodology, they estimate response error rates in 
panels of monthly labour force data from the Current Population Survey (CPS). The general 
consistency of the results is taken as an indicator that Markov Latent Class Analysis is a useful 
method to assess the accuracy of responses in the CPS. Critical to this analysis is confirming the 
Markov assumption; the authors present some interesting empirical evidence for its validity over the 
short term in the CPS. 

Many statistical offices use modified half-sample-replication (MHS) for estimating the sampling 
variance of medians. This is an important practical problem because direct calculation of sample 
medians can be computationally intensive. An alternative estimation method is to group the 
continuous data into discrete intervals and use linear interpolation over the interval containing the 
median. In their paper Thompson and Sigman compare the effects of no grouping (i.e., the sample 
median), grouping with fixed-size intervals, and grouping with data-dependent-sized intervals on 
medians and associated MHS variance estimates. Their empirical study shows that the data- 
dependent-sized intervals yielded variance estimates with the smallest bias, the best stability, and 
the best confidence intervals. 

McLaren and Steel consider the implications of different overlap patterns on the sampling 
variance of seasonally adjusted and trend estimates obtained from time series based on sample 
surveys by using the Census X-11 and X-11-ARIMA seasonal adjustment methods. They show that 
the “in for 8”, “in for 6”, “in for 4, out for 4, in for 4” rotation patterns are sensible if the one month 
change in seasonally adjusted estimates are the key statistics to be analyzed. If, however, the key 
statistics are the trend level and the difference between two consecutive trend estimates, then the “in 
for 1, out for 2, in for 1, for a total of 8 months” is a preferable rotation pattern to reduce the 
sampling variance. They also show that the “in for 2, out for 2, in for 2, for a total of 8 months is 
a reasonable compromise if the level and one months change in seasonally adjusted and trend 
estimates are both considered important. 

You and Rao present hierarchical Bayes multi-level models for small area estimation. The 
models allow random regression parameters that also depend on small area level covariates. The 
small area mean is estimated by the posterior mean and the posterior variance is taken as a measure 
of precision. Three variance models are considered: fixed equal, fixed unequal, and random. Details 
of Gibbs sampling for these models are presented and used for inference. Procedures are illustrated 
using county level household income data from Brazil. 
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In This Issue 


Okafor and Lee consider a two phase sampling scenario, where a subsample of the non 
respondents at the second phase are revisited according to a fixed sampling rate. Based on this 
scheme, modified versions of the ratio and regression estimators are suggested. Optimal values for 
the sample sizes and the fixed sampling rate aredetermined, based on cost functions, so as to 
minimize variance. In addition variances and their estimators are given. A small empirical study 
looks at the relative efficiencies of the modified ratio and regression estimators relative to the 
standard Hansen-Hurwitz estimator. 

Pickery and Loosveldt bring an important analytical technique to the study of item non-response. 
Their models present a more complete picture of the factors affecting item non-response than in 
previous work in this area. One important aspect of this approach is that the authors make a 
separation between interviewer/respondent specific variation, variation attributable to interviewer/ 
respondent characteristics and error variance. 

Fuchs investigates the affect that screen design and question order have on interviewer behavior 
in a Computer Assisted Interview (CAI) environment. Through the use of experiments under 
laboratory conditions, it has been shown that screen design and question order do affect interviewer 
performance. In his paper, Fuchs presents results from a field experiment which tests two different 
screen designs together with two different question orders in a 2x2 factor design. These results were 
based on time measures that were built into the CATI application and from 234 randomly selected 
interviews that were video taped and analyzed according to a coding scheme. 


M.P. Singh 
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Leslie Kish — A Life of Giving 


IVAN P. FELLEGI' 


1. INTRODUCTION 


I cannot believe that I am writing an article in memory of 
Leslie Kish. Just a few months ago I wrote a partly 
humorous little speech on the occasion of his 90" birthday 
celebration. I jokingly asked why are we making such a fuss 
about a 90" birthday — after all the Queen mother just 
celebrated her 100". I emphasized that that was something. 
He laughed heartily, with the well known “Kish twinkle” in 
his eye. I was struck once again by the extent to which he 
remained fun-loving, vibrant, insightful, in fact young in all 
aspects of behaviour — even if somewhat limited in his 
mobility. He told me about his forthcoming partial knee 
replacement operation and confided that his doctor told him 
that he will either undergo this operation, or he will need to 
use a walker to get around. Of course, a walker was not to 
be contemplated: he needed to have his full mobility. And 
mobility, at 90, meant not just getting around at home but 
traveling around the world several times a year. He died due 
post-operative complications, having fought for several 
weeks with his usual indomitable courage. 

In my mind the most characteristic feature of his life was 
his incessant giving. One of his last acts of giving was to 
inspire his friends and colleagues to establish the Leslie 
Kish International Fellows Fund to help students from 
developing countries obtain training in population 
sampling. 

Leslie was born in 1910 in Poprad, then part of the 
Austro-Hungarian Empire, now in Slovakia. He used to 
relate how, at various times throughout history, Poprad 
belonged to five different countries — an appropriate symbol 
of his life motivated by a love of people from all parts of the 
world. In 1925 his parents decided to migrate to the U.S.A 
— together with hundreds of thousands of other Hungarians 
who left their country. As the great Hungarian poet Attila 
Jozsef put it: “one and a half million of our people 
staggered out to America”. Soon after their arrival Leslie’s 
father died. The remaining family of mother and four 
children had to decide whether they will stay in the U.S.A. 
They did, but that meant that the two oldest children, 
including Leslie, who was then 16 years old, would have to 
work in order to help the others. 

Leslie continued his schooling in the evening. By !937 
he was within a year of completing his undergraduate 
studies. But this 27 year old was once again ready to 
sacrifice himself in order to help the world improve. He 
interrupted his studies in order help fight the fascists in 
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Spain as a member of the International Brigade. His love of 
things Spanish, and of people oppressed, stayed with him 
forever. 

At the end of the Spanish Civil War in 1939 he returned 
to the United States and completed his studies at City 
College of New York and received a degree in mathe- 
matics. He moved to Washington, where he was fortunate 
to have become a member of pioneering groups, first at the 
Bureau of the Census and then at the Department of 
Agriculture. 

Again, he interrupted his career to volunteer for service 
in the war. In 1947 he finally moved to the University of 
Michigan at Ann Arbor where, together with a small band 
of enthusiasts helped found the Institute for Social 
Research. He said later that he never worked as hard as he 
did in those early years: obtaining his M.A and Ph.D. while 
working full time but also finding time to teach. 

In statistics, he gave us several superb books. These 
include the pioneering Survey Sampling which became not 
just a bible of the field (7.e., like the original one, a source 
of lofty inspiration), as well as a day to day tool of practice. 
In that sense much of the world’s statistical system has 
embedded in it the hundreds of pearls of practical wisdom 
of Survey Sampling. In 1988 (when Leslie was a young 78) 
came Statistical Design for Research which integrated and 
organized a lifetime’s worth of acquired statistical wisdom. 
In between, before and after came a stream of articles, 
lectures and talks. He, sometimes working with others, 
introduced the concepts into our thinking and the words 
into our language of design effects; he was among the first 
to explore the issue of inference from complex samples and 
developed the innovation now known as balanced repeated 
replication (actually with Marty Frankel); was among the 
pioneers of studying response errors; became the apostle of 
rolling samples and censuses; pioneered controlled 
selection; formulated the concept of multipurpose designs; 
did some of the early work on small area estimation; and so 
on. But important as these works are, I think just as crucial 
were some of his other contributions. 

He was one of very few people whose early applied 
work made sampling respectable and admired. In addition 
to having been one of the founders of what became the 
Institute for Survey Research at Ann Arbor, he taught 
generations of statisticians, both Americans and foreign 
ones through the legendary Summer Program for Foreign 
Statisticians. After his formal retirement he continued to do 
so through lectures in the Summer Program; through 


Ivan P. Fellegi, Chief Statistician, Statistics Canada, 26" floor, section A, R.-H. Coats Building, Ottawa, Ontario, Canada K1A OT6. 
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decades of editing or contributing to one or another of the 
questions and answers columns of the Survey Statistician; 
and through numerous lectures and consulting assignments 
At international meetings I used to “bump into” his past 
students and current friends. One no longer “bumps into” 
them, because they have become completely ubiquitous: I 
wonder how many better known foreign samplers there are 
who were not at some point Leslie’s students. And I do not 
want to forget about two of my favourites among his many 
contributions. His years of faithful service to Statistics 
Canada as a founding member of our Advisory Committee 
on Statistical Methods; and his ASA presidential address of 
1977 (published in JASA in March 1978) — the best address 
that any President of ASA gave in my living memory. 

For his accomplishments he received world wide 
recognition. Of his dozens of awards I will just single out a 
few: he received an honorary doctorate from the University 
of Bologna on the occasion of its 900" anniversary, the 
Samuel Wilks Medal which is ASA’s highest recognition, 
the Henry Russell lectureship which is the highest reco- 
gnition of University of Michigan, the title Honorary 
Fellow of the ISI which I regard as a kind of Nobel price in 
statistics, and perhaps the most personally meaningful for 
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him: a slew of the highest possible recognitions from 
Hungary (honorary doctorate from the largest university in 
Budapest, honorary membership in the Hungarian Academy 
of Sciences and the Officer’s Cross of the Order of the 
Merit). 

Over and above what he gave us in statistics, he gave us 
the phenomenon known as “Leslie Kish, a force of nature”: 
the Spanish Civil War fighter, the philosopher of all things 
Statistical, the ever young agitator for human rights, 
raconteur, avid reader, author of the best annual Christmas 
letters, loving husband and father, and lifelong friend to 
hundreds, perhaps thousands. 

When I spoke at his 90" birthday celebration, I ended by 
saying that I was hoping to be present at Leslie’s really big 
anniversary — the one the Queen Mother had just passed. 
And that was not just a joke: he was so full of life, it was 
not only quite possible to contemplate him living to be a 
hundred, but rather it was impossible to think about the 
opposite. Unfortunately, he did pass away. His final act of 
giving was to donate his body to medical research. 
Wouldn’t it be fitting if the resulting work gave us some 
insight into the human wonder that was Leslie Kish?... 
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Population Size and Total Estimation When Sampling From 
Incomplete List Frames With Heterogeneous Inclusion Probabilities 


DAWN E. HAINES, KENNETH H. POLLOCK and SASTRY G. PANTULA! 


ABSTRACT 


Information from list and area sampling frames is combined to obtain efficient estimates of population size and totals. We 
consider the case where the probabilities of inclusion on the list frames are heterogeneous and are modeled as a function 
of covariates. We adapt and modify the methodology of Huggins (1989) and Alho (1990) for modeling auxiliary variables 
in capture-recapture studies using a logistic regression model. We present the results from a simulation study which 
compares various estimators of frame size and population totals using the logistic regression approach to modeling 


heterogeneous inclusion probabilities. 


KEY WORDS: Logistic regression; List frame; Area frame; Capture-recapture sampling. 


1. INTRODUCTION 


In this paper, we estimate population size and totals 
when information from multiple independent sampling 
frames is available. Population elements are assumed to 
have varying probabilities of inclusion for different 
sampling frames. These heterogeneous inclusion proba- 
bilities may depend on a covariate. For example, suppose 
we are interested in estimating the number of hog farms and 
the total number of hogs in North Carolina. Covariate 
measurements such as hog farm acreage or number of 
employees indicate the size of hog farms. Larger farms may 
have a higher chance of being included on a list frame than 
smaller farms. In capture-recapture experiments, animals 
may have unequal capture probabilities. Capture (inclusion) 
probabilities for animals may vary with respect to age, sex, 
size, or species. 

List frames are physical listings of sampling units in the 
target population. Items found on a list frame can include, 
but are not limited to, names, addresses, telephone numbers, 
social security numbers, or physical descriptions of loca- 
tions. These and other miscellaneous stratification variables 
are used to identify persons, animals, businesses, or other 
establishments. List and area sampling frames are con- 
structed and maintained to obtain estimates of the unknown 
population size and totals. Since frame imperfections such 
as omissions, duplications, and inaccurate recordings are 
inevitable in any large data collection operation (Hansen, 
Hurwitz and Madow 1953), various solutions for dealing 
with frame imperfections have been proposed in the 
literature. One approach, first developed by Hartley (1962, 
1974), combines an incomplete list frame with an area 
frame. Further theoretical extensions are due to Cochran 
(1965), Lund (1968), Fuller and Burmeister (1972), and 
Bosecker and Ford (1976). Haines and Pollock (1998a) 
apply the dual frame method to a bald eagle population 


while Haines and Pollock (1998b) present a more general, 
theoretical approach to combining multiple frames. These 
two papers do not consider the case where the inclusion 
probabilities are heterogeneous. Fienberg (1992) presents 
an annotated bibliography of the capture-recapture literature 
specifically related to the census undercount problem, 
including Wolter (1986, 1990), and Cowan and Malec 
(1986). 

The National Agricultural Statistics Service (NASS) 
currently employs a multi-frame approach for its sampling 
and estimation of numerous agricultural commodities. 
NASS collects and summarizes data on crop acreage, 
livestock, grain production and stocks, costs of production, 
farm expenditures, and other agricultural items. Fecso, 
Tortora and Vogel (1986) provide a review of sampling 
frames for the agricultural sector of the United States while 
Nealon (1984) details the multiple and area frame esti- 
mators used by the U.S. Department of Agriculture. 
Pollock, Turner and Brown (1994) offer a model-based 
capture-recapture solution for estimating frame size based 
on information from two incomplete list frames. According 
to Cochran (1977), it is often difficult to obtain a list that 
corresponds exactly to the population of interest. Lists 
routinely collected for some purpose are usually found to be 
incomplete, partially illegible, or to contain an unknown 
amount of duplication. Since list frames are typically 
incomplete, estimates based solely on list frames may 
underestimate the population size. Supplementing available 
information with an area frame sample may provide 
efficient estimates of the population size and totals. 

An area frame is a collection of geographical areas de- 
fined by identifiable boundaries. Area frames are often used 
by survey practitioners in order to attain complete coverage 
of the target population. Populations such as farms are 
naturally associated with the land units comprising the area 
frame. For example, in an agricultural survey, the region of 
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interest is divided into a set of geographic land masses 
called segments. Segments, which are the sampling units, 
are then selected using stratified multistage designs (Kott 
and Vogel 1995). Rules which link farms in the population 
to segments in the area frame are defined. Once the farms, 
or reporting units, within each sampled segment are 
identified, they are personally enumerated and the pertinent 
data collected. Nealon (1984) provides a detailed descrip- 
tion of the open, closed, and weighted segment estimators. 
Faulkenberry and Garoui (1991) formulate additional 
estimators specifically designed for area frames. More 
complex construction and sampling methods for area 
frames are discussed in Fecso et al. (1986). Area sampling 
and subsampling from area frames are considered in detail 
in Kott and Vogel (1995). 

In section 2, we consider independent list frames where 
the list frame elements have heteroscedastic inclusion prob- 
abilities. We discuss methods which provide population 
size and total estimators when information from list 
frame(s) and an area frame sample is available. Section 3 
summarizes results from a simulation study that compares 
various estimators of frame (population) size and popula- 
tion totals. Finally, results are summarized and discussed. 


2. HETEROSCEDASTIC INCLUSION 
PROBABILITIES 


2.1 Population Size Estimation with List Frames 


In capture-recapture experiments, different animals may 
have different capture probabilities. Similarly, individual 
elements may have different probabilities of inclusion on a 
list frame. Different list frames may be viewed as different 
. capture occasions. Model M, denotes the heterogeneity 
model in the closed population capture-recapture literature 
(Otis, Burnham, White and Anderson 1978). In a capture- 
recapture setting, capture probabilities, though assumed to 
vary from animal to animal, are assumed to be the same for 
all trapping occasions. The heterogeneity model may have 
up to N + 1 total parameters, namely N and p,,i = 1, ..., N, 
where JN is the population size and p, denotes the inclusion 
probability for the i-th unit. For multiple list frames, this 
corresponds to the assumption that the inclusion probability 
p, for element i is constant over all k list frames, 
Bi Bowinr Dy 

Burnham (1972) and Burnham and Overton (1978, 
1979) investigate the problem of estimating N in the 
capture-recapture setting. The proposed estimator for NV 
given by Burnham (1972) is based on the jackknife method 
of bias reduction (Quenouille 1956). Chao (1988) develops 
an alternative moment estimator for this model based on 
capture frequency data (Pollock 1991). Under certain con- 
ditions, Chao’s proposed estimator is less biased than 
Burnham’s jackknife estimator. In general, it is difficult to 
find a completely satisfactory estimator of N under Model M,,. 
Otis et al. (1978), as a result, suggest that one should design 


the entire study to minimize heterogeneity. Norris and 
Pollock (1996) propose a nonparametric MLE which is still 
not totally satisfactory. 

In capture-recapture experiments, the model expressed 
as Model -th allows inclusion probabilities to vary both 
by trapping occasion (list frame) and individual. Define p,, 
as the inclusion probability of the i-th element on the j-th 
list frame. Model M-th is obviously not easy to estimate 
since it can have up to tN + 1 parameters where ¢ = k, the 
number of list frames. Chao, Lee and Jeng (1992), using the 
idea of sample coverage, propose a nonparametric method 
of estimating the population size for Model -th. 

An alternative to the nonparametric approach is to model 
the inclusion probabilities as a function of an auxiliary 
variable. Pollock, Hines and Nichols (1984), Huggins 
(1989), and Alho (1990) address the role of auxiliary 
variables in capture-recapture experiments with unequal 
capture (inclusion) probabilities. The closed population 
capture-recapture experiments have i = 1, ..., N individuals 
and j =1,...,¢ trapping occasions. Again, the 7 = 1,...,¢ 
trapping occasions are similar to t=k, the number of 
independent list frames. Huggins (1989) and Alho (1990) 
propose a conditional estimation procedure for estimating 
the size of a closed population based on one capture and a 
single recapture. Both of these papers assume the logistic 
model for the inclusion probabilities, given by 


exp (a, + B;x;) 


ye exp(a, + B,x,) 


(1) 


ij 


where x, is a covariate a, and 6, are unknown parameters. 
Note that this parameterization yields 0 < p, < 1 for all 
values of aL, and B.. For B.>0, the inclusion probability 
increases with the covariate. This parameterization is 
different from the probability proportional to size (pps) 
sampling where Py is assumed to be proportional to x,. 
The MLEs of a, and B, can be obtained using the like- 
lihood conditioned on the unit being on at least one list 
frame. Haines (1997) derives the conditional likelihood 
function for three independent list frames. 

Treating each individual as a separate stratum, define the 
following indicator variables for 7 = 1, ..., NV: 


1 individual i belongs to frame j only 
Uu,, = 
” |0 otherwise 
(ay 
and 
1 individual i belongs to both frames 
de — 
" {0 otherwise. 
The value of the expression 
Mj = Ujp + Ui, + 4; (2) 


is one if individual 7 is included on at least one of the two 
frames and zero otherwise. 
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Alho (1990) presents the conditional likelihood function 
for two list frames as 


exp{ay.Np, +0, .N, + Bs, Dee 


Ty, - K,@) 


Xi $ Be duae, x; 


>» (3) 


where 


K,(@) = exp (dy, +B, x,} + exp (ay, + By x,) 


+ Xp { Op as MB; "7 (By. ar By); 


and 0 = (a, ; B. uss Bs. )’. Alho (1990) uses an iterative 
procedure based on the sufficient statistics to maximize (3) 
while we implement Newton’s method to calculate condi- 
tional MLEs of 6, denoted 6 = (Gg, Ba, de, ; Bs,)’. See 
Appendix A of Haines (1997) for details on Newton’s 
method. The estimated probability that individual 7 is 
included on at least one list frame is denoted 


pl tial ban si 
1 +exp(a, +B; x) 
] rs 
Sa ete |b 9050) 0 ¢ min iC 4) 
| 1 * exp(G y ea 
where 


exp(a, + Box) 


a7. 1 +exp(a, = Bixps 


=1,..,Nand j = B,, B,. (5) 


If © were known, the Horvitz-Thompson estimator of 
Nis N = Yu, -1 1/x, (Horvitz and Thompson 1952). From 


Cochran (1977), the variance of N is 


(6) 


i 


An estimate of the variance of N is 


2 
M,=1 Tl; 


Since ® is unknown, we consider the population size esti- 
mate given by N= My Vass 1/% 7 ,,, where Tt, is defined in (4). 
An estimate of the variance of N 1S derived using Taylor’s 
method and has the form 


where (7) 
on, (6) 


4 1 
M,=1 is a6’ 


NS 
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and )\(6) is the inverse of the Hessian matrix. The second 
term in (7) is due to estimating 7,(@) by 7,(8). 

Another population size estimator commonly used in 
Capture-recapture experiments is the Lincoln-Petersen 
estimator. This classic estimator is due to Lincoln (1930) 
and Petersen (1896) and has the form 


_Ns,Np, 
L-P N 


A 


where N, and N, denote the size of list frames B, and 
B, eecuyery, aad N,,, denotes the number of units 
common to both fenres ’This is a simple method of 
moments estimator based on the assumption that all units 
have homogeneous inclusion probabilities for each of the 
two independent list frames. It is possible for the denomi- 
nator N,, 5, 10 be zero. Chapman (1951) proposed a modi- 
fied version of the Lincoln-Petersen estimator, given by 


(Np, a 1)(N 5, =o 1) 


Noy = - 1. (8) 
CH Ce et 


This estimator is less biased than the Lincoln-Petersen esti- 
mator (Chapman 1951). According to Sekar ~ Deming 
(1949), the asymptotic standard error of Now 


where NV, and N, denote the number of units belonging 
only to list fre B, and B,, respectively. 

The Lincoln- Betemen eancin is the unconditional 
maximum likelihood estimator of the population size N 
when there are two independent list frames and the 
inclusion probabilities are homogeneous. Haines (1997) 
extends the estimation procedures to k list frames, each 
with homogeneous inclusion probabilities. This estimator, 
however, is not appropriate when the inclusion probabilities 
are heterogeneous. See the simulation results in section 3. 


2.2 Population Size Estimation with Area and List 
Frames 


Suppose we have access to an area frame in addition to 
two list frames, B, and B,. The area frame consists of U f 
segments that cover the entire population. A simple random 
sample of u, segments is selected. We assume that all units 
in the sampled segments are observed. The probability of 
inclusion in the area frame sample is the same for all units 
and is the known quantity p, = u,/U,. Next, we maximize 
the conditional likelihood (3) with respect to 8 and calcu- 
late the estimated probability that individual 7 is included 
on at least one list frame or the area frame. This probability 
is denoted # , = %, + p,(1 - 7,). The probabilities z , and p P, 
are defined in (4) and (5), respectively. An estimated 
Horvitz-Thompson estimator for population size is 
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Ni igy paisa 9) 
iésample 7 i 
This estimator can easily be extended to the case with k list 
frames, B,, ....B,, and an independent area frame. Fe 
From Cochran (1977), an estimate of the variance of NV 
is given by 


7AS AL (10) 
where A is defined in (7) and yi is the inverse of the 
Hessian matrix. The variance formula for NW in (6) and its 
estimate are valid only when 1,,, the probability that units i 
and / are included in the sample, is equal to 1, 7,. When an 
area frame sample is included, however, 2, is not 
necessarily equal to 1,7, Suppose units 7 and / belong to 
the same area frame segment. In this case, units 7 and / are 
both included or not included in the sample, depending on 
whether their corresponding segment is selected or not. It 
can be shown that the joint inclusion probability, 7,,, can be 
estimated as 


if i and/ belong to different 


TT, 
area segments 


if 7 and/ belong to the same 
area segment (11) 


Py +4 ,7,(1 -P,) 
where 7 , is defined in (4) and z , =, + p,(1 - 7,). Hence, 
when i and / belong to the same segment, a A 
However, if p , 18 small and T, and i, are large, then 
_ (&,,- % , 7 ,) will be close to zero. 


! 


2.3 Population Total Estimation with List Frames 


Suppose y, values are available for all elements on two 
independent list frames B, and B,. If 8 were known, an 
estimate of the population total, Y, is the Horvitz- 
Thompson estimator 


yj 


Vile Bl (12) 
ae ~, 7, (8) 
According to Cochran (1977), the estimated variance of 
Y-7 18 
2 
tes y/ (1 - 20) 
Vit dead ys Ta 
M,=1 Tt; (8) 


When @ is unknown and is estimated by 6, an estimate for 
the population total is 


An estimate of the variance of Y y-t iS derived using 
Taylor’s method and has the form 


D n~ 
5 3 yj (10) Mees Me 
A ge) 8 OLS 
M,=1 1; (8) 
where ‘ 
y, O7, (0) 


B= 


M71] 726) 06" 


and y(6) is the inverse of the Hessian matrix evaluated at 6. 
These ideas extend easily to incorporate k independent list 
frames. 

In practice, y,’s may not be observed for all units on the 
list frames. Consider the case where y,’s are available for 
only a random sample of n B, and n B, units from list frames B, 
and B,, respectively. By construction, the inclusion ani! 
bilities, Pip Vary with the individual 7 and frame 7. How- 
ever, once individuals are included on a list frame, they are 
subsampled using simple random sampling. As a result, all 
elements on list frame B, have equal chance (n B, /N, ) of 
inclusion in the subsample. Note that we are Felecia 
samples from each list frame rather than drawing a single 
sample from a combined list frame. Since the list frames are 
assumed to be independent, the estimated probability the 
i-th individual is included on at least one of the two list 
frames is 


Ea Bs eae Amigo att aie Iie 4, 
i Pisa, VW PN, SEE Ny Noe ( ) 


An estimated Horvitz-Thompson estimate of Y is obtained 
by substituting (14) into (12). 
Another estimator of the population total, Y, in this case 


is 
ie VCH Pe “ , 

N,, i N,, i Np,b, 
which is Chapman’s estimator multiplied by the mean of the 
responses for those elements included on at least one list 
frame subsample. Again, this estimator is valid only when 
the inclusion probabilities are homogeneous. There are 
Ny tN, is unique elements in frames B, and B,. A 
pais SO Sinihe can be defined when informaeonn is ae 
able only for subsamples from the list frames. 


2.4 Population Total Estimation with Area and List 
Frames 


Consider the case where, in addition to y, values for the 
units on the list frames (or subsamples from list frames), y, 
values are available for all elements in a random sample of 
segments from an area frame. Inclusion of the area frame 
information results in the estimated inclusion probability for 
the i-th individual, namely 
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i, > T, +p =a); (15) 


where 7t , is defined in (4) or (14), depending on whether y, 
is observed for all units on the list frames or only for a 
subsample of units, respectively. An estimated Horvitz- 
Thompson estimator of the population total in this case is 


a J; 
AS : (16) 


ie¢sample 7, 


An estimate of the variance of Y H-T IS given by 


7 y, (1 ~%,) 
40 eae »S : ; i 
M, = T 
(Xi, T, T,) , 
+2) > + yy, + BY B’. a7) 
Be eg ip TH Me 


where 7,, is defined in (11) and B and > are defined in 
(13): 


3. SIMULATION STUDY 


3.1 Assumptions of the Study 


To study the properties of population size and total 
estimators, Haines (1997) considered eighty different 
models. Details for only two of those models are presented 
here. One assumption made is that the inclusion probabi- 
lities for two list frames depend on a covariate x,. 
Secondly, we assume that the covariate may be correlated 
with the response variable y,. Also, we assume that x, and y, 
are lognormally distributed with correlation p.. The 
lognormal distribution is utilized which allows for a skewed 
distribution of covariates. We generate x, as e “' and y, as e "', 
where u, and v, are generated as bivariate normal random 
variables with zero means, unit variances, and correlation 9, ,. 
It can be shown that p,, = log[p,,(e-1) + 1]. 

Consider a population of size N. Assume that there are 
two independent list frames, B, and B,, and an area frame, A. 
The area frame is assumed to be complete in the sense that 
it covers the entire population. A sample of area frame 
segments is selected and the units within each area segment 
are observed. Let p, denote the inclusion probability for 
any element to be included in the area frame sample, where p , 
is assumed to be the same for all individuals. 

The probability that the i-th element is included on the 
j-th list frame is given by the logistic regression model (1) 
for i=1,..,.N and j = B,, B,. We assume the probability 
that the i-th element is included on list frame B, is inde- 
pendent of its inclusion status on list frame B, and the area 
frame sample. 
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3.2 Parameter Settings 


We consider various parameter values. For the popu- 
lation size, N, we take N=3000r1,000. We use 
P,, = 0.3, 0.0, 0.5, and 1 corresponding to negative, zero, 
positive, and perfect correlation between the response 
variable and the covariate. Here, p, = 1 corresponds to 
x, =y;, indicating that the inclusion probability is directly 
related to the response variable. 

For each of the above 2x 4 = 8 parameter settings of NV 
and Pry» We consider two models corresponding to different 
choices of ag,B3,03, and B,. Recall that E(x,) = 
E[e “| = e°>. Consider an element with covariate value 
given by the mean value e°°. The probability that this 
element is included on the j-th list frame is 


exp (a, +B, e°° 
p= p (a, B, ) Be ye 
1 + exp (a, +B, eo) 
If a, = -B,e°°, then this element has a 50% chance of 
being included on list frame 7. We use this relationship in 
Model 1. 
Extending the above idea, if we set 


Pon 0.5 

ca ey Bie, 

then the unit with mean covariate value has probability p of 
being included on list frame 7. If we assume that the 
inclusion probabilities are the same for list frames B, and B,, 
then the chance of being included on at least one of the two 
list frames is given by 1 - (1 - p)*. This relationship is used 
in Model 2. Specific values of a, and B, for the two models 
are summarized in Table 1. 


a, = log ( 


Table 1 
Summary of Model Parameters 
1eGlep,”). 
Model a B a Blea aety ee ; 
B, B, B, B, B, B, ad “Pz,” 
1 0 0 0 0 0.5 0.5 0.75 


2 -0.5478 0.8 -0.5478 0.8 0.6838 0.6838 0.90 


For each of the 2 x 4 x 2 = 16 models, we consider three 
p, values given by 0, 0.05, and 0.20. Here, p, =0 
corresponds to using only the information from list frames B, 
and B,. 


3.3 Generation of the Data 


For each of the above sixteen models, we first generate (x, y,) 
using the bivariate lognormal distribution for 7 = 1, ..., N. 
We then “generate” (identify) the units that belong to list 
frames B, and B,. We use the probability p,, to include the 
i-th element on list frame j. Finally, using p, = 0.05, we 
identify the elements belonging to area frame A. We repeat 
the process for the case p, = 0.20. For each parametric 
combination, we generate 1,000 Monte Carlo replications. 
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3.4 Estimators 


For population size, we consider Chapman’ s estimator, Ne. 
given in (8). This estimator assumes that B, = Ba = = 0 and 
does not utilize the information from the area frame sample. 
We also consider the estimated Horvitz-Thompson esti- 
mators discussed in section 2. 

For estimating the population total of a response 
variable, we consider the case where the response is 
observed for all list frame elements. Elements in an area 
frame are sampled with probabilities p, =0,0.05, and 
0.20. We do not consider population total estimates based 
on subsamples from each list frame. The population total 
estimate, 1G , has the same form as (16) with 7, defined in 


(15). Stary the population size estimate, N,. , has the 
same form as (9). 
The estimator 


Yous.p, = New ¢p,» p, = 9, 0.05, 0.20 

is also considered where Yop) is the sample mean of the 
y,’s included in the “sample.’ The performance of YQ, “) 
is dependent on Nae which was observed to underestimate 
N considerably for Model 2. The results for sae are not 
included here. Another design-unbiased Bae of ¥ is 


given by 
A J; 
ie€"area sample" PD A 


This is the Horvitz-Thompson estimator based on the area 
frame sample alone. Since complete enumeration of area 
segments is expensive, p, is typically small in practice. For 
small p,, Y, is expected to have a much larger variance 
than Y_ since the estimator Y_ includes information from 
list frames in addition to information from the area frame 
samples. Hence, results for Y , are not included. 


3.5 Estimated Variance of the Estimator 


In our simulation study the values of p, considered are 
very small. In contrast, the probability of inclusion on at 
least one of the list frames is large for each individual. As 
a result, 7, is close to 7, and 7, in (11) is close to £, it,. 
Hence, the second term in equations (10) and (17), 
involving 7,, - 7, %,, are expected to be small. We have not 
included this term in our estimate of the variance. Despite 
this omission, we observe that the estimated variance is 
very close to the empirical variance of the estimator for the 
models we consider. 


3.6 Summary Statistics 


For the population size estimates, we present results 
averaged over the 4,000 replications corresponding to the 
four values of p. i and 1,000 Monte Carlo replications for 
each p... For each model, we summarize the mean and 
standard deviation of the estimates, average of the estimated 
standard errors of the estimators, the percent relative root 


mean square error (% RRMSE), and the empirical coverage 
probabilities of a 95% confidence interval. These measures 
are all standardized by the population size N. We report 
results for Models 1 and 2 in Tables 2 and 3, respectively. 


Table 2 
Population Size Estimates for Model 1 
N = 300 N CH N 0 N 0.05 N 0.20 


Average of estimates divided by N 0.999 1.011 1.007 1.004 


Standard deviation of estimates 


vided by 0.059 0.077 0.059 0.048 
Average of estimated standard 

deviation of estimator divided by VN %-959 9-072 0.059 0.047 
Jo RRMSE 0.003 0.006 0.004 0.002 
Coverage 0:947 0195S) 90.957 0:950 
N= 1,000 


Average of estimates divided by N 1.000 1.003 1.002 1.002 


Standard deviation of estimates 


cividedoy yi 003.1 i K0!035590:030 10/025 
Average of estimated standard 
deviation of ccuntor divided’ by Nie 70a lm a a 
% RRMSE 0.001 0.001 0.001 0.001 
Coverage 0.954 0.959 0.958 0.956 
Table 3 
Population Size Estimates for Model 2 
N= 300 Neu No Noos _No.20 
Average of estimates divided by N 0.922 1.006 1.005 1.003 
Standard deviation of estimates 
divided by N 0.032 0.052 0.049 0.040 
Average of estimated standard 
deviation of estimator divided by V Bas LR Ness Ae 
% RRMSE 0.007 0.003 0.002 0.002 
Coverage 0.271 0.953 0.954 0.951 
N=1,000 — 


Average of estimates divided by N= 0.921 1.001 1.001 1.001 


Standard deviation of estimates 
divided by N 


Average of estimated standard 
deviation of estimator divided by NV 


Jo RRMSE 


Coverage 


0.018 0.028 0.027 0.022 


0.015 0.027 0.026 0.021 


0.007 0.0008 0.0007 0.0005 
0.009 0.949 0.949 0.949 


Similarly, for the population total estimates, we present 
summary Statistics averaged over the 1,000 replications 
corresponding to each parametric combination. We summa- 
rize the mean and standard deviation of the estimates as 
well as the average of the estimated standard errors of the 
estimators, where the estimates are scaled by the true total 
(Y) for that replicate. In other words, for each replicate we 
divide the estimate by its replicate total, Y. We then 
compute the mean and the standard deviations of these 
standardized estimates. Similarly, for each replicate, we 
compute the estimated standard error of the total estimator 
divided by the total for the replicate and then compute the 
average of these standardized values. We report these 
because the totals change from replicate to replicate. 
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Finally, we report the coverage probabilities of the 95% 
confidence intervals for the total. Results for Models 1 and 
2 are respectively presented in Tables 4 and 5. 


3.7 Conclusions 


3.7.1 Population Size Estimation 


In Model 1, the inclusion probabilities do not depend on 
the covariate. In this case, Chapman’s estimator N,,, is 
very Close to the maximum likelihood (Lincoln-Petersen) 
estimator and hence is expected to perform better than Np. 
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The estimator Ny loses efficiency since it estimates the 
parameters Qs B By p> and B Bp» which have the value zero 
in this model. The estimator N, os has about the same effi- 
ciency as Ne . The bias in all the estimates is minimal. For 
Model 1, Me notice that the average of the estimated 
standard deviation is close to the standard deviation of the 
estimates. This indicates that the standard error estimate we 
use performs well. Also, we notice that the empirical cover- 
age probabilities are all within three standard errors of 0.95. 
That is, all of the empirical coverage probabilities are 


within (0.95 + 3 [0.95 x 0.05/4,000]*) = (0.94, 0.96). 


Table 4 
Population Total Subsampling Estimates Scaled by Y for Model 1 
N= 300 N= 1,000 

P,, Ne YY pea 1 T- ean Nea Coats ch MD 
-0.3 Average of estimates 1.004 1.003 1.001 1.002 1.002 1.001 
Standard deviation of estimates 0.077 0.073 0.062 0.042 0.040 0.035 
Average of estimated standard error 0.076 0.072 0.061 0.041 0.039 0.033 
Coverage 0.953 0.951 0.949 0.946 0.942 0.942 
0 Average of estimates 1.013 1.012 1.008 1.001 1.001 1.001 
Standard deviation of estimates 0.080 0.070 0.059 0.041 0.039 0.033 
Average of estimated standard error 0.081 0.072 0.060 0.040 0.038 0.033 
Coverage 0.951 0.954 0.951 0.944 0.942 0.946 
0.5 Average of estimates 1.053 1.018 1.009 1.004 1.003 1.002 
Standard deviation of estimates 0.586 0.104 0.072 0.057 0.045 0.037 
Average of estimated standard error 0.233 0.094 0.070 0.051 0.045 0.036 
Coverage 0.950 0.951 0.945 0.950 0.955 0.955 
1.0 Average of estimates 1.064 1.030 1.013 1.013 1.009 1.006 
Standard deviation of estimates 0.515 0.162 0.090 0.094 0.066 0.047 
Average of estimated standard error 0.277 0.128 0.086 0.070 0.059 0.046 
Coverage 0.930 0.929 0.930 0.946 0.949 0.951 

Table 5 
Population Total Subsampling Estimates Scaled by Y for Model 2 
N= 300 N= 1,000 

0 ald Yu eum 0 Bee Sel pee al Xe 
-0.3 Average of estimates 1.010 1.009 1.006 1.003 1.003 1.002 
Standard deviation of estimates 0.098 0.092 0.078 0.052 0.049 0.041 
Average of estimated standard error 0.094 0.089 0.074 0.051 0.048 0.040 
Coverage 0.935 0.926 0.931 0.952 0.955 0.955 
0 Average of estimates 1.008 1.007 1.005 1.002 1.002 1.001 
Standard deviation of estimates 0.065 0.062 0.050 0.034 0.032 0.027 
Average of estimated standard error 0.064 0.061 0.051 0.034 0.032 0.028 
Coverage 0.953 0.950 0.952 0.947 0.951 0.955 
0.5 Average of estimates 1.002 1.002 1.001 1.001 1.001 1.001 
Standard deviation of estimates 0.035 0.033 0.028 0.019 0.018 0.915 
Average of estimated standard error 0.035 0.034 0.029 0.019 0.018 0.016 
Coverage 0.954 0.950 0.951 0.965 0.967 0.965 
1.0 Average of estimates 1.001 1.001 1.001 1.000 1.000 1.000 
Standard deviation of estimates 0.021 0.020 0.017 0.012 0011 0.010 
Average of estimated standard error 0.021 0.020 0.017 0.012 0.011 0.009 
Coverage 0.952 0.949 0.954 0.947 0.947 0.943 
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For Model 2, the inclusion probability is a function of the 
covariate. As a result, Nas is not an appropriate estimator 
for NV. We observe that N,, significantly underestimates 
the true population size. On the other hand, Ne provides 
a good estimate of N. The bias in N, decreases as p, 
increases in Model 2. Further, the relative bias decreases as 
the population size increases. i! 

As expected, the standard deviation of NV decreases as 
the area frame inclusion probability p, increases. For 
example, in Model 1 where N = 300, the inclusion of a 5% 
area frame sample reduces the relative standard deviation 
from 0.077 to 0.059, a 23% reduction. When a 20% area 
frame sample is utilized, the relative standard deviation 
decreases from 0.077 to 0.048, a 38% reduction. When 
N= 1,000, the inclusion of a 5% area frame sample 
decreases the relative standard deviation from 0.035 to 
0.030, a 14% reduction. Increasing the area frame sample 
to 20% reduces the relative standard deviation from 0.035 
to 0.025, a decrease of 29%. Generally speaking, the rela- 
tive standard errors decrease as population size increases. 
Although the average of the estimated standard error of 
N_ is smaller than the empirical standard deviation, the 
difference is relatively small. Also, the coverage probabi- 
lities of the 95% confidence interval based on NV are very 
close to 0.95. In contrast, the coverage probabilities of the 
95% confidence interval based on N,,, are 0.271 and 0.009 
for N = 300 and N = 1,000, respectively. r 

Based on our simulations, we recommend the use of V 
with a large value of p,. The choice of p, is determined in 
practice by area frame sampling costs, which are not taken 
into consideration in our study. 


3.7.2 Population Total Estimation 


For population totals, we observe results that are very 
similar to what we observed for the population size. In gen- 
eral, relative biases and standard errors decrease as p, in- 
creases and as the population size increases. We also notice 
that the average relative estimated standard error is very 
close to the empirical standard deviation of the standardized 
estimator standardized by the total. This suggests that the 
approximate standard error formula in (7) is a good estimate 
of the standard error. Note also that the empirical coverage 
probabilities are mostly within three standard errors of 0.95. 
That is, most of the empirical coverage probabilities fall 
within (0.95 + 3[0.95 x 0.05/1,000]”) = (0.929, 0.971). 


4. SUMMARY 


In this paper, we studied the performance of the esti- 
mated Horvitz-Thompson estimator of the population size 
and total based on samples from area and list frames. We 
presented methods for estimating the parameters of the 
logistic regression model for the inclusion probabilities. 
Though numerous models and other estimators are 
considered in Haines (1997), we presented simulation study 
results for only two models and a few estimators. 


We believe the methods used in this paper are potentially 
very useful to survey researchers because list frame 
incompleteness is a fact of life. Our results are among the 
first to suggest a method of estimating population totals 
which account for incompleteness and model the inclusion 
probabilities as a function of the covariates. 
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An Estimation Method for Nonignorable Nonresponse 


JEAN-FRANCOIS BEAUMONT' 


ABSTRACT 


When a survey response mechanism depends on a variable of interest measured within the same survey and observed for 
only part of the sample, the situation is one of nongnorable nonresponse. In such a situation, ignoring the nonresponse can 
generate significant bias in the estimation of a mean or of a total. To solve this problem, one option is the joint modelling 
of the response mechanism and the variable of interest, followed by estimation using the maximum likelihood method. The 
main criticism levelled at this method is that estimation using the maximum likelihood method is based on the hypothesis 
of error normality for the model involving the variable of interest, and this hypothesis is difficult to verify. In this paper, 
the author proposes an estimation method that is robust to the hypothesis of normality, so constructed that there is no need 
to specify the distribution of errors. The method is evaluated using Monte Carlo simulations. The author also proposes a 
simple method of verifying the validity of the hypothesis of error normality whenever nonresponse is not ignorable. 


KEY WORDS: Nonignorable nonresponse; Maximum likelihood; Estimation equations; Regression imputation; 


Reweighting. 


1. INTRODUCTION 


When a survey response mechanism depends on a 
variable of interest measured in the same survey and 
observed for only part of the sample, the situation is one of 
nonignorable nonresponse. In measuring income, for 
example, it may be realistic to assume that low income 
earners will exhibit a lower tendency to respond than high 
income earners, or vice versa. Readers will find in Little 
(1982) a formal definition of the concept of nonignorable 
nonresponse. In such a situation, ignoring the nonresponse 
can generate significant bias in the estimation of a mean or 
of a total. To solve this problem, one option is the joint 
modelling of the response mechanism and the variable of 
interest, followed by estimation using the method of 
maximum likelihood, used for example in Greenlees, Reece 
and Zieschang (1982), and imputation of the missing 
values. The main criticism levelled at this method is that 
estimation using the method of maximum likelihood is 
based on the hypothesis of error normality for the model 
involving the variable of interest, and this hypothesis is 
difficult to verify. 

Rancourt, Lee and Sarndal (1994) described simple 
correction factors aimed at reducing the bias generated by 
nonresponse that is not ignorable without reference to the 
hypothesis of normality and in the absence of a response 
mechanism model. These correction factors, however, are 
only available for ratio imputation. 

In this paper, the author proposes an estimation method 
that is robust with respect to the hypothesis of normality, so 
constructed that there is no need to specify the distribution 
of errors. The author also proposes a simple method of 
verifying the validity of the hypothesis of error normality 
whenever nonresponse is not ignorable. 


In section 2, the problem is defined and some notation is 
introduced. In section 3, various estimators of the mean of 
a population are introduced for a variety of hypotheses 
concerning the response mechanism and the distribution of 
data. In section 4, an estimation method is proposed for 
nonignorable nonresponse. In section 5, the author 
describes the results of a simulation study used to compare 
the estimators described in the two preceding sections. 
Finally, the last section contains a brief discussion. 


2. NOTATION 


In the following, we attempt to estimate the mean of a 
variable Y for a certain population P. To do so, we select a 
sample S, and the variable Y is observed for only part of the 
sample. The sample of respondents is denoted R, and the 
sample of nonrespondents is denoted O. We assume that 
there is at least one variable that is observed for all the 
sampling units and correlated with Y. 


The estimator of the mean, p =)» ¥,/N, where N is 
the size of the population, can be obtained by weighting the 
respondent units: 


3 es ema amar: G1) 


where w, denotes the sampling weights that correspond to 
the inverse selection probability and Wr ; denotes the 
weights that correspond to the estimated inverse response 
probability. Another estimator of the mean can be obtained 
by imputing the missing values: 
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ie Dee w,Y, + Ee wT, (2.2) 


Dies ei 


where Y, denotes values that are imputed for the non- 
respondent units. 

For the sake of simplicity, we assume, in the following, 
that the sampling weights are constant for all units of the 
population. Thus, we can eliminate w, from equations (2.1) 
and (2.2). We also assume that there is only one observed 
variable for all sampling units. This variable is denoted_X. 


3. CURRENT ESTIMATION METHODS 


In this section, equations (2.1) and (2.2) are developed 
under a variety of hypotheses concerning the response 
mechanism and the distribution of data, and appropriate 
estimation methods are described. In section (3.1), we 
assume a uniform response mechanism; in section (3.2), we 
assume a response mechanism that depends on_X, while in 
section (3.3), we assume a response mechanism that 
depends on Y. The response mechanisms in sections (3.1) 
and (3.2) are ignorable, whereas the one in section (3.3) is 
not ignorable. 


3.1 Uniform Response Mechanism 


Assuming a uniform response mechanism, we have the 
same response probability for all sampling units. Thus, 
estimator (2.1) becomes: 


io i€R Y; 
Sy ache re (3.1) 
Np 
where n, is the total number of respondents. This 


estimator is the very same one we would have obtained by 
using equation (2.2) and by imputing the respondent mean 
for all nonrespondents. 


3.2 Response Mechanism Dependent on X 


When the response mechanism depends on variable X 
(correlated with Y), estimator (3.1) might be strongly 
biased. It is then preferable to use this variable as addi- 
tional information for the estimation of mean p. 

Estimator (2.1) can be obtained by replacing 1 / Wr ; by 
the estimated response probability using a logistic regres- 
sion. A response probability model is therefore needed. If 
we only have one observed variable (X) for all sampling 
units, the model can be written as follows: 


1 


PCRS tastes at ae ae ah 
a ; 1 + exp[-(dy + 0,X;)] 


where a, and o., are parameters to be estimated (using the 
maximum likelihood method, for example) and R, is a 


dichotomous variable equal to 1 if unit i responds and to 0 
otherwise. The estimator of the mean obtained in this way 
is denoted 1p y. 

If we prefer to use estimator (2.2) instead, the missing 
values can be imputed using the following model: 

¥; = By + BX, + €, (3.2) 
where B, and B, are unknown parameters and ¢, is a 
random error term of zero mean that is not correlated with X,. 
The imputed values are given by: Y, =By + B, X,, where By 
and B,’ are estimates (obtained by means of the method of 
least squares using the respondent units) of By and B, 
which are in turn estimates of B, and B,. In fact, B, and 
B, are the estimates that would have been obtained (using 
the method of least squares) if we had observed all the units 
of sample S. The estimator obtained in this way is denoted 1 x 
Note that all the models considered in this document are 
assumed to be valid for all the units of sample S. 

We could also add a residual to the imputed values in 
order to better estimate the variance due to sampling (see 
for example Gagnon, Lee, Rancourt and Sarndal 1996). 
However, this technique still does not make it possible to 
estimate the variance due to imputation. Moreover, it tends 
to produce estimates of the mean that are less precise than 
if no residual had been added. Since this paper does not 
deal with variance estimation, we have chosen not to add 
residuals to the imputed values. This has the added 
advantage of simplifying the calculation of p x 


3.3. Response Mechanism Dependent on Y 


All the estimators of the mean discussed so far can be 
strongly biased when the response mechanism depends on 
Y (nonignorable response mechanism). For such a response 
mechanism, the response probability can be modelled as 
follows: 


1 


PCR ee een 
1 + exp[-(a, +4, ¥,)] 


(3.3) 


Since variable Y is only observed for respondent units, it is 
impossible to obtain an estimate for a, and a, using the 
maximum likelihood method. Model (3.2) can also be 
used. However, the parameter estimates will not be 
consistent since E(€,|R,=1) and E(¢,X,|.R,=1) are not 
zero. Even if we had consistent estimates, the missing 
values could not be imputed as described in section (3.2) 
since E(Y,|R,=0,X,)#B)+B,X; (Greenlees, Reece and 
Zieschang 1982). If, for example, the response probability 
correlates positively with the variable of interest Y, then, for 
a given value of X, the mean of nonrespondent units will be 
lower than that of respondent units, and will therefore be 
lower than the mean of all units taken together. A similar 
argument can be presented if the response probability 
correlates negatively with the variable of interest. In fact, 
it can be shown that 
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cov(Y,,p(¥,) LX,) 


E(Y¥,1R,=0,%,) = By + 1 -E(p(¥,)1X,) ” 


BX; im 


where p(Y,) = PCR, =ANY,): 


The two approaches in section (3.2) are therefore invalid 
when the response mechanism is not ignorable. In such a 
situation, a better approach would be to estimate the para- 
meters of models (3.2) and (3.3) simultaneously. The 
method of maximum likelihood can be used to this end. 
This method, however, requires as an additional hypothesis 
that errors €, follow a normal distribution (or any other 
distribution relevant to the type of data analyzed) with 
constant variance o”, and that they be mutually 
independent. The natural logarithm of the likelihood 
function / can be written as follows: 


1= Vice MIP(Y) AY 1X) 


ieee In B (poy), )HvaG'4) 
where f(Y,|X,) is the probability density function of e 
normal distribution with a mean By + B, X; and variance ov 
The method of maximum likelihood consists in finding the 
parameter values which maximize /. To carry out the maxi- 
mization, it must be possible to approximate E(p(Y,)1X;). 
This can be achieved by using a numerical integration 
method similar to that of Greenlees, Reece and Zieschang 
(1982). In this paper, the following approximation (Zeger, 
Liang and Albert 1988) has been used instead: 
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ERVIN eee asa perme nae ee oc eee emerges (6 
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where k = L/yc?o? a; +1 and c = 16¥3/15z. This ap- 
proximation is based on the hypothesis that errors follow a 
normal distribution with constant variance. This approxi- 
mation was preferred to a method of numerical integration 
because it is simpler and computationnally faster, an advan- 
tage that must be considered seriously before any simula- 
tion study is undertaken. Finally, equation (3.4) was maxi- 
mized using the Newton-Raphson algorithm and the NLIN 
procedure of the SAS software (SAS Institute Inc. 1990). 
Once the parameters of models (3.2) and (3.3) have been 
estimated, estimators of the mean (2.1) or (2.2) can be 
chosen. Estimator (2.1) is obtained by replacing Wri ; by 
1/p*(Y,), where p*(Y,) is the estimated response proba- 
bility. This estimator is denoted pp y yy - Estimator (2. 2) 
can be obtained by determining imputed values Y,” in such 
a way that )).,e, is minimized and that the constraints 
Lies @; = 0 and ik, e,X,=0 are met, where e, = Y, 
=e BX, for ieR, e.=Y, —B, -B,X,, for ic O, and 
B, and B; are the estimates of B, and B, respectively. The 
estimator of the mean can then be written as follows: 


Hiya =Bo + Bi VicsX;/n, where n is the size of sample S. 
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The reasoning behind this approach is that the two previous 
constraints would have been met if variable Y had been 
observed for all units in the sample and if this variable had 
been modelled using model (3.2). 


4. PROPOSED METHOD OF ESTIMATION 


This section describes the proposed method of estimation 
for a nonignorable response mechanism (section 4.1), as 
well as a graphic method (section 4.2) that can be used to 
verify the error normality hypothesis of model (3.2). 


4.1 Method of Estimation for a Response 
Mechanism Dependent on Y 


The method of maximum likelihood is valid when errors 
exhibit a normal distribution and have the same variance. 
When this hypothesis does not hold, it is preferable to use 
a more robust method of estimation. 

If response probabilities p(Y,) were known and greater 
than zero for all sampling units, a robust method of estima- 
tion (in terms of both the error normality hypothesis and 
model 3.2) would consist in minimizing the error sum of 
squares weighted by the inverse response probability 
p(Y,). This minimization is equivalent to solving the 
system of prey, 


ae p( : ) 


where Z,, = 1 and Z,, = X,. This approach is considered 
robust with respect to the normality hypothesis since the 
method of least squares does not require that the distribu- 
tion of errors be specified. Weighting by means of the 
inverse response probability also provides a certain robust- 
ness in terms of model (3.2). In fact, estimators By and B, 
obtained using equation (4.1) are consistent with respect to 
the response mechanism for B, and B, (which are the 
estimators of B, and B, that would have been obtained if 
there had been no nonresponse) regardless of the validity of 
the model. A similar argument may be found in Sarndal, 
Swensson and Wretman (1992, p. 519), but in terms of the 
sample selection mechanism instead of the response 
mechanism. 

Likewise, if the probability density function /(Y, |X,) 
was known (not necessarily normal and yet not dependent 
on the parameters of model 3.3), we could then estimate 
parameters a, and a, of model (3.3) using the maximum 
likelihood method, for example, and solve the system of 
sacar 


Dee a 2 infp(y,)] 


——(Y,-B,-B,X,)Z,=0, k=1,2, 41) 


+D 5 Sin -E@(Y)IX)]=0, 4.2) 
1€ da, 


fork =Oandk=1. 
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Thus, the estimates of parameters B,, B, , O, anda, are 
obtained by solving the unbiased estimation equations (4.1) 
and (4.2). An algorithm that can be used to find the solu- 
tion consists in solving alternately the systems of equations 
(4.1) and (4.2) until convergence is achieved. This requires 
the possibility of calculating E(p(Y,)|X,) in equation 
(4.2). However, this last expectation requires that the distri- 
bution of errors ¢, be known, and in all likelihood it is 
unknown. To get around this problem, we must use an 
approximation, and a number of them can be considered, 
including approximation (3.5). Another option would be to 
develop a strategy based on the bootstrap method by 
selecting the respondent units proportionally to their inverse 
response probability. However, this method requires consi- 
derable computer processing time, and is not considered in 
this paper. Instead, we have chosen the following approxi- 
mation, obtained by linearizing p(Y,) using a Taylor series 
assessed at point E(Y,|X,) and by taking the expectation 
of the first two terms in this series: 

E(p(¥,)1X;) = p(E(Y,1X;)) = Pp(By + BX). (4.3) 

It should be noted that the expectation of the second term 
in the series is zero. This approximation offers the advan- 
tage of requiring only the first moment of the distribution of 
Y, conditional on X;. In this sense, it should be robust with 
respect to the error normality hypothesis since it does not 
require that the error distribution be specified. Of course, 
if the distribution of errors is known or can be properly 
estimated, it will be possible to find better approximations 
than (4.3) although, in this case, it may be preferable to use 
the maximum likelihood method. 

Another interesting property of approximation (4.3) is 
_ that alternately solving the systems of equations (4.1) and 
(4.2) might be achieved using the following algorithm: 


1. determine initial values for the response probabilities 
(or for parameters a, and a, ), e.g., let p(Y,) = 1 for 
all the respondent units; 


2. letj = 1, where 7 is the number of iterations; 


3. solve the system of equations (4.1) by means of the 
current response probability estimates, p(Y,)/~), 
using a weighted regression procedure to obtain Be : 

Y). 
and B,"; 

4. impute the missing values using Y, x ) = BY + BY We for 
TeO 

5. solve the system of equations (4.2) by using a logistic 
regression procedure to obtain p( y,) J 


6. stop once convergence has been achieved, otherwise let 
j =j + 1 and return to step 3. 


It is sufficient then to simply have a linear regression 
procedure and a logistic regression procedure to obtain the 
desired estimates. This algorithm is a very efficient means 
of finding the solution although, in certain cases, many 
iterations might be needed before convergence is achieved. 


In actual practice, it did converge in all cases where it was 
used. It should also be noted that this algorithm shows 
certain similarities with the EM algorithm used by 
Dempster, Laird and Rubin (1977), except that here we do 
not maximize a likelihood function. 

For the simulations in the next section, we selected 
instead the Newton-Raphson algorithm which converges 
more rapidly. However, the above-mentioned algorithm 
had to be used for the few cases in which the Newton- 
Raphson algorithm met with convergence problems. 

The proposed algorithm might be very useful as a means 
of providing initial values for a more rapid algorithm such 
as the Newton-Raphson one. The proposed algorithm could 
simply be used with a not very demanding convergence 
criterion so that, after only a few iterations, it could provide 
sufficiently good initial values to ensure convergence of the 
Newton-Raphson algorithm. In a different context, 
Beaumont and Demnati (1998) used a similar approach by 
beginning the iterative process using an algorithm of the 
EM type so as to provide the initial values for a more rapid 
algorithm of the Newton-Raphson type. They were able to 
show empirically that the combination of the two algo- 
rithms represents a sound compromise between processing 
time and efficiency in finding a solution. 

As in section (3.3), once the parameters of models (3.2) 
and (3.3) are estimated, we can select estimators of the 
mean (2.1) or (2.2). Estimator (2.1) is obtained by replacing 
Wri by 1/p*(¥,), where p *(Y,) is the estimated response 
probability. This estimator is denoted 1p y gog. Estimator 
(2.2) is also obtained as in section (3.3) by determining the 
imputed values Y,” in such a way that )°_, e, is minimized 
and the constraints )’,., e, = O and )’,., e, X, = 0 are met, 
where e, = Y,-B, -B,X,, for i€R, and e, = 
Y, -B) -B,; X,, for icO. This estimator is denoted 
Hy. Rog: The quality of these two estimators of the mean 
will depend largely on the validity of models (3.2) and (3.3) 
and on the quality of approximation (4.3). 

A modification of step (5) for the algorithm presented in 
this section was proposed by Beaumont (1999). The results 
of a simulation study show that this modification provides 
results that are slightly better than those obtained using the 
method proposed in this paper. However, this no longer 
involves using the maximum likelihood method to estimate 
the parameters of model (3.3), given that f(Y,|X;,) is 
known and a logistic regression procedure can no longer be 
used for step (5). It should nevertheless be mentioned that 
it is not absolutely necessary to use the method of maximum 
likelihood to estimate a, and a, , although it is the method 
preferred in this paper. 


4.2 Verifying the Error Normality Hypothesis 


In order to use the method of maximum likelihood, we 
might be interested in verifying the error normality hypoth- 
esis (or rather the residual normality hypothesis since the 
errors are not observed). In the absence of nonresponse, a 
traditional method (D’ Agostino 1986, p. 25, equation 2.11) 
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consists in producing the graph of ®! [F n(é,)] in terms of 
residuals e,, for icS, where @(.) is the distribution 
function for a random variable having the standard normal 
distribution, and F(.) is the empirical distribution 
function. Whenever errors exhibit normal distribution, the 
points in this graph should more or less fall along a line 
having a slope 1/o passing through the origin. 

If there is nonresponse, the same strategy can be used as 
in the previous paragraph, but the empirical distribution 
function must be estimated using the respondent units. 
Since the units in the sample respond with unequal proba- 
bilities, the estimated empirical distribution function can be 
given by the formula (Sarndal, Swensson and Wretman 
19925 p. 199): 


Die. et e,<e, I/p*(¥;) 
Dy ACS) 


Note that, in this last equation, the response probabilities 
are estimated as opposed to the Sarndal, Swensson and 
Wretman formula, in which selection probabilities are 
known. Thus, the error normality hypothesis can be 
verified by producing the graph of ®'[F, (e,)] in terms of 
residuals e,, for i¢R. This method will be valid provided 
that F, (e) can correctly estimate F\(e,), as is the case 
when the response probabilities are correctly estimated. 
When the nonresponse is not ignorable, and when the 
method of estimation proposed in this paper is used, the 
response probabilities should be properly estimated if 
models (3.2) and (3.3) are appropriate along with 
approximation (4.3). 


F, (e,) = 


5. SIMULATION STUDY 


In order to compare the estimators of the mean presented 
in the two previous sections, we carried out a simulation 
study. We simulated 4 populations with a size of 1,000 
according to model (3.2) with B, = 2 and B, =3. Random 
variables X, are independent of one another and they follow 
an exponential distribution of mean 1. Errors ¢, are inde- 
pendent of one another, are not correlated with the X, and 
have a mean of zero and a variance o”. In two populations, 
the errors follow a normal distribution (¢, ~ Nor(0,07)), 
and in the other two populations, the errors follow an 
exponential distribution of mean o recentred at 
O(e,~ Exp(o)-o). For each of these distributions, one 
population has a standard deviation o equal to 1.5 corre- 
sponding to a squared coefficient of correlation (between XY 
and Y) of 80% (R? = 80%), and the other has a standard 
deviation equal to 3 corresponding to a square coefficient 
of correlation of 50% (R? =50%). 

For each population, we simulated 1,000 samples of 
respondents according to model (3.3) with a, =0.5. 
Parameter a, was determined separately for each of the 4 
populations, so that the mean response rate would be 70%. 
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This parameter varied between -1.185 and -0.958. Note 
that we have here a census (n = N = 1000). The advantage 
of this is that we can concentrate solely on the nonresponse 
error since there is no sampling error. Moreover, the fact 
that populations of relatively large size (1,000) are gener- 
ated makes it possible to emphasize the bias of the esti- 
mators instead of their variance, since the variance should 
diminish as the size of the population increases (for a fixed 
mean response rate). 

For each of the 1,000 samples of respondents, we calcu- 
lated the 7 estimates of the mean described in the two 
previous sections. We then calculated, for each population, 
the mean and the variance of these 1,000 estimates, denoted 
jp’ and S fs respectively. Finally, we calculated an estimate 
of the relative bias (expressed as a percentage), RB* = 
{(a@* -p)/u] x 100%, an estimate of the standard error 
associated with this relative bias, SEs— 


(100/ 1) /S°./1 000, and an estimate of the root mean 
square errors, RMSE* = /S,: + (* -p). 


The results of the simulation study are shown in Table 1. 
An analysis of this table indicates that, regardless of the 
error distribution, the relative bias and the mean square 
error of all the estimators is lower when the correlation 
between X and Y is greater, which is not surprising. 


Table 1 
Simulation Results Used to Compare 7 Estimators of the Mean p 
R? = 80% R? = 50% 
Estimator : 3 \ G r A 
RB(%) SE RMSE RB(%) SE RMSE 
Population with normally distributed errors 
Upu 16.90 0.03 0.84 26.68 0.04 1.33 
Mx 5.65 0.02 0.28 18.02 0.03 0.90 
eran -0.14 0.03 0.05 CZ 010 O17 
tan 1.14 0.03 0.08 10.12 0.06 0.51 
Bh y 5.50 0.02 O27" “INT 0.03 0.89 
tia O13" 20.03 104) 1.03 10.07 =" 043 
a a 0.64 0.03 0.05" ~7.53 0.06" "0.39 
Population with exponentially distributed errors 
yu} Fs 17.83 0.04 0.86 26.60 0.05 1.29 
ne . 5.44 0.02 0.26 16.06 0.04 0.78 
* -0.54 0.02 0.04 5.18 0.05 0.26 
Mp y,Mz 
epee 1.31 0.021 70.07 VOY 2743890 030.36 
Mix 5.19 0.02 0.25 15.41 0103; FRE O75 
i -3.42 0.03 0.17 -25.47 0.05 1023 
Hy Y,ML 
: 0.49 0.02 0.04 4.07 0.03 0.20 
Hy, ROB 


An analysis of the relative bias indicates that the method 
of maximum likelihood provides best results when the 
errors are normally distributed, followed by the robust 
estimation method described in section (4.1). Estimators 
which assume a nonignorable response mechanism have a 
lower relative bias than those which incorrectly assume an 
ignorable response mechanism. Among the latter esti- 
mators, the most biased is estimator p eae For a given 
method, there is generally little difference between the 
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weighted estimator (2.1) and the estimator that includes 
imputed values (2.2). However, the latter must be given a 
slight advantage. 

The conclusions in the previous paragraph always apply 
when errors are exponentially distributed, except that the 
robust estimation method becomes the best. This observa- 
tion should not be surprising since the method of maximum 
likelihood is based on the error normality hypothesis. How- 
ever, the weighted estimator pp yj, remains slightly 
biased, and this is more difficult to explain. 

The conclusions drawn from an analysis of the relative 
bias still apply when analyzing the mean square error. In 
fact, estimators which are very biased show a strong 
tendency to having a high mean square error and vice versa. 


6. DISCUSSION 


When the hypothesis of a nonignorable response 
mechanism is realistic, and when the hypothesis of error 
normality for linear regression model (3.2) is justified, 
using the method of maximum likelihood may be appro- 
priate. However, when the latter hypothesis is not justified, 
the results of the simulation study described in section 5 
show that the robust estimation method presented in this 
paper is preferable. 

Moreover, Beaumont (1999) described the results of 
another simulation study indicating that the estimation 
method proposed in this paper is robust with respect to both 
the error normality hypothesis and model (3.2). As for the 
method of maximum likelihood, it has been shown to be 
even more sensitive to the validity of model (3.2) than to the 
hypothesis of error normality. The latter method should 
therefore only be used when all the hypotheses associated 
with models (3.2) and (3.3) are reasonable. 

Obviously, all estimators show little bias when non- 
response is very low. Likewise, when the coefficient of 
correlation between X and Y is very high, all estimators 
show little bias, except for the estimator which assumes a 
uniform response mechanism pp y. In either case, the 
choice of an estimator should be based on the criterion of 
simplicity, which favours the estimators in section (3.2), 
specifically estimator p x 

It should be noted that models (3.2) and (3.3) could be 
complexified according to the nature of the problem. For 
example, other independent variables could be included in 
these models. Variable Y could also be categorized using 
dummy variables, and these dummy variables could be used 
in model (3.3) instead of variable Y itself. 

In this paper, we have dealt only with the problem of the 
estimation of a mean when the response mechanism is not 
ignorable. However, the methods described in sections 3 
and 4 apply to other types of estimation. For example, 
weights or imputed values could be used for the estimation 
of parameters in a given regression. 


This paper has attempted to describe a robust estimation 
method with respect to the hypothesis of error normality for 
model (3.2), making it possible to reduce the bias due to a 
nonignorable response mechanism. In some future work, it 
would be interesting to evaluate simple methods of variance 
estimation using imputed data and this robust estimation 
method. 
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An Approximate Design Effect for Unequal Weighting 
When Measurements May Correlate With Selection Probabilities 


BRUCE D. SPENCER' 


ABSTRACT 


It is common practice to estimate the design effect due to weighting by 1 plus the relative variance of the weights in the 
sample. This formula has been justified when the selection probabilities are uncorrelated with the variable of interest. An 
approximation to the design effect is provided to accommodate the situation in which correlation is present. 


KEY WORDS: Weighting; Deff; Sampling variance; Complex samples. 


1. INTRODUCTION 


It is common practice to weight observations in an 
unequal probability sample by the reciprocals of selection 
probabilities. The rationale is that failure to use the weights 
will cause bias if the sampling weights correlate with the 
variable of interest. A drawback to weighting is an increase 
in sampling variance when the weights vary excessively in 
the sample. This increase may be quantified by the design 
effect. The design effect is the ratio of the variance of the 
Statistic of interest under the design of interest to the 
variance of the statistic under simple random sampling with 
the same sample size (Kish 1965). Design effects are 
important both for approximating standard errors after the 
sample is in hand and for predicting standard errors ahead 
of time, which is critical for efficient design of samples. 

Kish (1965, 1992) discussed an approximation for the 
design effect for weighted estimates from unequal proba- 
bility samples: 1 + rvw, with rvw defined as the relative 
variance of the weights in the sample. Thus, if w, is the 
weight of unit 7 in the sample and w is the sample mean, 
rvw =n ').”_(w,-w)’/*. Gabler, Haeder, and Lahiri 
(1999) used a superpopulation model to derive a design 
effect when clustering is present as well. Their formula, 
which agrees with design-based results in Kish (1965), 
reduces to 1 + rvw when there is zero intraclass correlation. 
The 1 + vw approximation for the design effect is based on 
a model or design in which the weights are uncorrelated 
with the variable of interest (and hence an unweighted 
estimate would serve as well or better than the weighted 
estimate). Here we develop an approximation to the design 
effect under a model in which correlation may be present. 
In developing the approximation we do not assume that the 
population is sampled from a superpopulation. The accu- 
racy of the approximation depends only on the charac- 
teristics of the sample design and the population of interest. 

For simplicity, we will discuss single-stage unequal 
probability sampling with replacement. Heuristic extension 
of the results to sampling without replacement is indicated 
in section 4. 


2. REGRESSION REPRESENTATION OF 
POPULATION AND SAMPLE DESIGN 


Let y, denote the measurement of interest, P, the (draw- 
by-draw) selection probability for a sample of size n, and 
w, = 1/(nP,) the sampling weight for unit i in a population 
of size, 1 < i< N. Observe that P = eel pPplNie NG - 
Consider the least-squares population regression line 


Vesa BP + s, (1) 


with a = Y-BIN, B= Ee i7,-Y@, PY: (P= P)P and 
Y= si ,¥;/N. Denote the population variances of the y’s, 
the €’s, the £°” S, and the w’s by 3, O., o,, and oO. , with, for 
example, 5, =yv NO -Y)/N. Denote the population corre- 
lation between y and P by p, p, between € and w by p,,,, 
and between ¢* and w by p,,. It follows from the 
properties of least- “Squares, Or equivalently from the defi- 
nitions of o and f, that Y,¢,P,=)).,e,/N=0 and o; = 
(ithe p;, BG, If data are available, we can fit the regression 
representation (1) and estimate a, B, o, and p by, say, 
a, B, 6, and 9. 

Let Y 27 -|W,y;, denote the usual weighted estimator of 
the population total, Y. The variance of ¥ is well-known 
(Cochran 1977, 253) to be 


N 
Ven Ley Pha (2) 
i= 
Using the regression formulation (1), we may re-express 
the variance as 


V(Y) = 02N(W-Nin)+(1 - p;, p)S, NW+Np,2,6,20, 


+2aNp, © & 6,» (3) 
where W =)" w,/N. 

This expression does not rest on any assumptions about 
the fit of the regression model. (See section 5 for deri- 
vation). 

If the regression model fits well enough so that p,2 ,, and p, ,, 
are zero, then the variance in (3) simplifies to VY) = 

a2N(W - Nin) + (1 - P;, p)S, NW. If simple random sampling 
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with replacement had been used, the variance would have 
been n-'N’o. Therefore, if pz, and p, , are negligible, 
the design ttc is approximately 


deff = (1 -p,, p)nWIN + (alo,)*(nWIN - 1). (4) 


This approximation does not require that the residuals 
from the regression are negligible, and it can hold wheno, 
is large. A referee has pointed out that the condition that 
p,2,, and p, ,, are negligible may seem unnatural in a model 
that regresses y on P rather than on w«1/P. Note, however, 
that if we had not only zero correlation between ¢ and P but 
also independence, then we would have zero correlation 
between functions of € and functions of P, and so p,2 ,, and p, ,, 
would be zero as well. 


3. ESTIMATION OF DESIGN EFFECT 


To estimate the design effect after the sample is in hand, 
we may use | + rww to estimate nW/N. To understand the 
rationale for this, note first that 


ee (5) 


The expectation of the numerator is NW/n. The expecta- 
tion of w is N/n, and so the denominator of (5) may be 
taken as an estimator of (N/n)’. Dividing the expectation 
of the numerator by (N/n)’, we obtain nWIN. Thus the 
design effect may be estimated from the sample by 


(1 - By, (1 + rvw) + (4/6,°(rww). (6) 


As a special case, note that if we set p, , = 0, the case of 
“haphazard weighting” (Kish 1992), then the estimate of 
the design effect simplifies to 


1 + rww + rw (#/6,). (7) 


This estimate is close to Kish’s approximation when 
a/6, is near zero. 


4. SAMPLING WITHOUT REPLACEMENT 


To derive the exact design effect for sampling without 
replacement would be more complex, as it would require 
consideration of joint selection probabilities for pairs of 
units. A heuristic extension of the results is easy, however. 
Recall that the ratio of the variance of a sample mean under 
simple random sampling without replacement to the 
variance under with-replacement sampling is approximately 


(1 - n/N). 
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The results we have derived for the design effect will 
apply to single-stage unequal probability samples of n units 
without replacement if the variance of the Horvitz- 
Thompson estimator of the total is approximately (1 -n/N) 
times the variance in (2), with P, taken as ' times the 
overall selection probability for unit i (Sarndal, Swensson, 
and Wretman 1992, 154). 


5. DERIVATION OF VARIANCE FORMULA (3) 


From (2) we have V(Y) =n "(y™, yi IP, - Y’). Next, 
note that (1) implies that 
2 = (Na+) = Na? + 2NaB + B? (8) 


and 


N N 
Yo y7/P, => [02/P, + BP, +€//P, + 2a + 2ae,/P, + 2Be,] 
i=l i=l 


N N N 
=0?)> P,'+B?+)> &/P,+2NaB+2a)- ¢,/P, 


i=1 i=1 i=1 


=O ny w. “Pande ew, +2NoB “2a ew,. (9) 
Subtracting (8) from (9) and dividing by 7 yields 


A N N N 
VP) =| > w,-N2in| +> &w, +20), ew,. 
i=l i=l i=l 


To obtain (3), note that 


N 
2 
> €;w, = NPp,2,,0,20 +NWo. 


w ew 
i=] 


=Np,2 ,6,26,, + (1- p;, p)6, NW 


and 


N 
Dt, E,W, =NQoy 6, G,,- 


i=l 
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On the Validity of Markov Latent Class Analysis for 
Estimating Classification Error in Labor Force Data 


PAUL P. BIEMER and JOHN M. BUSHERY' 


ABSTRACT 


The primary goal of this research is to investigate the validity of Markov latent class analysis (MLCA) estimates of labor 
force classification error and to evaluate the efficacy of MLC analysis as an alternative to traditional methods for evaluating 
data quality. We analyze interview data from the Current Population Survey (CPS) for the first three months of each of three 
years — 1993, 1995, and 1996 — and conduct an additional analysis of the CPS unreconciled reinterview data for 
approximately the same time periods. The reinterview data provides another approach for estimating CPS classification 
error that, when compared with the MLC estimates, helps to address the validity of the MLCA approach. Five dimensions 
of MLCA validity are addressed: (a) model diagnostics, (b) model goodness of fit across three years of CPS, (c) agreement 
between the model and test-retest reinterview estimates of response probabilities, (d) agreement between the model and test- 
retest reinterview estimates of inconsistency, and (e) the plausibility of patterns of classification error. In addition, we 
consider the robustness of the MLCA estimates to violations in the Markov assumption. Our analyses provides no evidence 
to question the validity of the MLC approach. The method performed well in all five validity tests. 


KEY WORDS: Panel surveys; Nonsampling error; Unemployment; Data quality. 


1. INTRODUCTION 


The Current Population Survey (CPS) is a household 
sample survey conducted monthly by the U.S. Bureau of the 
Census to provide estimates of employment, unemploy- 
ment, and other characteristics of the general U.S. labor 
force population. National estimates from the CPS of the 
size, composition, and changes in the composition of the 
labor force are published each month by the U.S. Bureau of 
Labor Statistics in Employment and Earnings. The CPS 
labor force estimates comprise one of the Nation’s key 
economic indicators; since 1942, the Federal government 
has used the CPS data series to monitor month-to-month 
and year-to-year changes in labor force participation. 

Given the importance of the CPS data series to public 
policy, there have been numerous evaluations of the accu- 
racy of the data. For example, since the early 1950s, the 
Census Bureau has conducted the CPS Reinterview 
Program to evaluate the quality of the labor force data. The 
program involves drawing a small subsample (less than 5 
percent) of the CPS respondents and re-asking some of the 
questions asked in the original interview — particularly the 
labor force questions. Until 1994, about one fourth of the 
sample received an unreconciled reinterview and three 
fourths received a reconciled reinterview. The reconciled 
reinterview component, which was used primarily for inter- 
view quality control purposes, was discontinued in 1994 
due to concerns about the quality of the data. However, the 
unreconciled reinterview continues today and is used to 
estimate the test-retest reliability (or response consistency). 
Forsman and Schreiner (1991) provide a detailed descrip- 
tion of the CPS Reinterview Program. 


1 
U.S.A. 


Several papers prepared by researchers outside the 
Census Bureau analyze the CPS Reinterview Program data 
to estimate the classification error in the CPS (cf Sinclair 
and Gastwirth 1996, 1998; Biemer and Forsman 1992; 
Chua and Fuller 1987; Poterba and Summers 1986; Abowd 
and Zellner 1985). Recently, Poterba and Summers (1995) 
used data from the CPS Reinterview Program to estimate 
the CPS classification error rates and to evaluate the impact 
of classification error on labor market transition rates. As in 
the 1986 paper, their more recent analysis is based on the 
assumption that the CPS reinterview reconciliation process 
yields data which may be considered as the truth. Abowd 
and Zellner (1985) took similar approach. 

Several authors (viz., Sinclair and Gastwirth 1996, 1998; 
Biemer and Forsman 1992; Forsman and Schreiner 1991; 
Schreiner 1980) question the assumption that reconciled 
reinterview yields true values. They provide considerable 
evidence that the reinterview data are subject to substantial 
classification errors. In fact, this realization was respon- 
sible for the Census Bureau’s decision to eliminate the 
reconciled reinterview portion of the CPS Reinterview 
Program in 1994. 

As an alternative to the infallibility assumption, Chua 
and Fuller (1987) and Fuller and Chua (1985) apply a type 
of latent structure model to the CPS reconciled reinterview 
data to estimate the CPS response probabilities. For model 
identifiability, they impose tight restrictions on the response 
probabilities, forcing the bias due to classification error to 
be zero for both interview and reinterview. In addition, they 
assume independent classification errors for the interview 
and reinterview (referred to as the ICE assumption in the 
literature) and across the months in sample. The ICE 
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assumption is a limitation of their analysis because evidence 
in the literature suggests that the assumption may not hold 
for the CPS (see, for example, O’ Muircheartaigh 1991, and 
Singh and Rao 1995). Consequently, response probabilities 
estimated using the Chua and Fuller approach may be 
biased. 

Sinclair and Gastwirth (1996) and Sinclair and Gastwirth 
(1998) apply a latent class modeling approach to the CPS 
interview-reinterview data using model restrictions origi- 
nally proposed by Hui and Walter (1980) for medical 
diagnostic testing. Using the interview-reinterview data 
cross-classified by sex, Sinclair and Gastwirth assume that 
classification error probabilities are equal for males and 
females while labor force participation rates differ for these 
groups. Since the model parameters consume all the avail- 
able degrees of freedom for parameter estimation, no resi- 
dual degrees of freedom are available to test model lack-of- 
fit. Consequently, their analysis does not directly address 
whether these model assumptions hold for the CPS data. 

In an examination of the determinants of rotation group 
bias, Shockey (1988) also applies latent class analysis to the 
CPS. His analysis suggests that the rotation group bias 
problem first reported by Bailar (1975) may be caused by 
response error arising from the interview administration. 
Shockey did not use reinterview data but rather relied on 
confirmatory factor analytic methods to support his claims. 
The sizes of his error rates were much larger than those 
reported by other authors which may be an indication of 
model bias. Unfortunately, like Sinclair and Gastwirth, 
Shockey’s data set is not adequate to test fully the 
assumptions of the model he used. 

The method of Markov latent class analysis, a promising 
approach for estimating the classification error in panel 
survey data, previously has not been applied to the CPS. 
' This method takes advantage of the repeating nature of 
panel surveys to extract information on classification error 
directly from the interview data. The MLCA model is 
really a combination of two models: a latent Markov chain 
model representing the month to month transitions among 
the true labor force classifications and a classification error 
model representing the deviations from the true and 
observed labor classifications. 

Because MLCA takes advantage of the repeating nature 
of panel surveys to extract information on classification 
error directly from the interview data, it does not require 
external, infallible measurements or rerneasurements 
obtained by reinterview methods. In that regard, the 
method offers some advantages over both the Census 
Bureau’s traditional methods and the methods of Chua and 
Fuller, Abowd and Zellner, Porterba and Summers, and 
Sinclair and Gastwirth for evaluating survey data quality in 
surveys. In many panel surveys, reinterviews are not 
feasible due to budget constraints, field work complexity, 
and respondent burden. MLCA may be the only way to 
assess the measurement error in these surveys. For panel 
surveys, such as the CPS, where reinterview data are 


available, the reinterview and MLCA methods offer alter- 
native analytical approaches for evaluating classification 
error. For example, as in the present analysis, MLCA can 
be used to model and test the traditional reinterview 
analysis assumptions. Further, MLCA analysis provides a 
statistical framework for combining the panel data and 
reinterview data to obtain even more information about 
classification error (van de Pol and Langeheine 1997). 

Another advantage of MLCA is the potential for incor- 
porating the entire panel data set into the estimates of classi- 
fication error rather than only the relatively small sample 
selected for reinterview. As a result, a number of data 
quality issues for panel surveys that previously could not be 
explored for lack of data may now be tractable. 

This paper reports our findings regarding the utility of 
the MLCA modeling approach for evaluating labor force 
classification error in the CPS. Software for fitting a wide 
variety of MLCA and other latent class models is available 
from several sources. The software employed in our 
analysis is 0EM (Vermunt 1997), which can fit a large class 
of log-linear models with or without latent variables. The 
flexibility and generality of this software allow the 
measurement error analyst to test a considerable range of 
classification error models and to explore hypotheses 
regarding the causes and correlates of classification error. 

In the next section, we describe the MLCA model and 
estimation methodology and its theoretical underpinnings. 
In section 3, we develop the MLCA methodology for the 
CPS application, fit a series of models to the CPS, and 
examine the fit of these models. In this section, we also 
produce estimates of classification error based upon the best 
MLCA model. In section 4, we conduct a number of tests 
of the validity of the MLCA estimates including a compari- 
son of the MLCA estimates with those from new interview- 
reinterview analysis. Finally, in section 5, we summarize 
our findings and make recommendations regarding the 
utility of the MLCA method for future evaluations of labor 
force classification error. 


2. MARKOV LATENT CLASS ANALYSIS 
FOR THREE TIME PERIODS 


Markov latent class models were first proposed by 
Wiggins (1973) and refined by Poulsen (1982). Van de Pol 
and de Leeuw (1986) established conditions under which 
the model is identifiable and gave other conditions of esti- 
mability of the model parameters. In this section, we devel- 
op the MLCA model in the context of the CPS and suggest 
other applications and its generalizations. 

Let the CPS target population be divided into Z groups 
(such as age, race, or sex groups) and let the variable G be 
the label for group membership. For example, G, = 1 if the 
i-th population member is in group 1, G, =2 for group 2 
and so on. Let X ei? Ye ei? and Z,, denote the true labor 
force classifications for the i-th person in group 
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G=g(for g=1,..., 2 and i=1 


as 


HO () ) where X. is defined 


1 if person (g, i) is employed 
in time period 1 
2 if person (g, 7) is unemployed 
8! in time period 1 
3 if person (g, i) is not in the labor force 
in time period 1 


with analogous definitions for Y,, and Z,, for periods 2 
and 3 respectively. Let yal denote Pr CRS Xy TARY, 
ee zlG=g), letx,,,, denote Pr(Y =y|X =x,G =g) and let 
, denote Pr(Z =z =z|Y=y,X=x,G=g). Then, the proba- 
bile? ‘that an individual in group g has labor status x in 
period 1, y in period 2, and z in period 3 is 
Ne yzig  Mxig Myig,x zigx,y° (1) 
Finally, under the first order Markov assumption, a neces- 
sary condition for model identifiability (see Van de Pol and 
de Leeuw 1986), we assume 


m - Nig, y (2) 
i.e., at period 3, the true status of an individual does not 
depend on the period 1 status, once the period 2 status is 
known. An alternate interpretation is that the current status, 
given the prior period’s status, does not depend upon the 
prior period’s transition. 

One can conceive of a number of scenarios where the 
Markov assumption may not hold for monthly labor force 
status. The assumption would be violated, for example, if 
individuals who are unemployed in period 2 are more likely 
to be unemployed in period 3, given they were also 
unemployed in period 1. The group of people unemployed 
in period 2 and period 1 probably includes a higher propor- 
tion of chronically unemployed people than the group 
unemployed in period 2 but not in period 1. That group 
(unemployed period 2, not period 1) likely contains a higher 
proportion of people temporarily out of work while 
changing jobs. 

However, the validity of this assumption cannot be 
adequately explored using the observed data because the 
data are distorted to some unknown extent by the presence 
of classification errors. At least two methods for assessing 
the validity of the Markov assumption for panel data are 
available. Van de Pol and de Leeuw (1986) suggest a 
method based upon four waves of panel data that substitutes 
a second order Markov restriction for the first order restric- 
tion in (2). Another method, suggested by van de Pol and 
Langeheine (1997), uses a combination of labor force panel 
data and the reinterview data at each time period. Neither 
of these methods was employed in this paper to test the 
MLCA assumption directly. Instead, we assessed the 
overall validity of the MLCA estimates using the methods 
discussed in section 3.2 below. In section 3.6 we provide 


Zz1g,x,y 
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some results from a simulation study to illustrate the robust- 
ness of the MLCA estimates of classification error to viola- 
tions of the Markov assumption. 

Now, consider the observed fait: force classifications 
from the CPS denoted by A,; and Ci for periods 1, 


2, and 3, respectively, where. *e 


1 if person (g, 7) is classified as employed 
in time period 1 
2 if person (g, i) is classified as unemployed 
gi | in time period 1 
3 if person (g, i) is classified as NLF 
in time period 1 


with analogous definitions for the response indicators, Bi 
and Cy for periods 2 and 3, respectively. Using an exten- 
sion of the notation established above, we denote the 
response probabilities in each of these classifications as 
Nip =Pr(A =alG=g, X=x), with analogous definitions 
for x,,,, and m,,, .. Thus, 7, _ Lies 22 is the probability 
that the CPS classifies a person in group g as employed 
(A = 1) when the true status is unemployed (X = 2). 
Likewise, 7, _ dig.x=2 is the probability that the CPS 
correctly classifies a person in group g as unemployed. 
Finally, we assume 


us 


a,b,clg,x,y,z  Malgx ™bigy *clge (3) 


or that classification error in the observed labor forces 
Status is independent across the three months. This assump- 
tion, referred to as the local independence assumption, has 
been investigated for the CPS by Meyers (1988) in his 
review of the Abowd and Zellner (1985) estimation 
approach. Meyers concluded that the assumption “seems a 
reasonable approximation.” Singh and Rao (1995), who 
studied the robustness of the assumption under a number of 
labor force population scenarios, reached a similar conclu- 
sion. Van de Pol and Langeheine (1997) modeled the joint 
distribution of panel data and reinterview data using latent 
class models to test for local independence for various types 
of labor force transitions. They found some evidence that 
people who change labor force status have lower reliability 
than those who do not, however the effect was quite small. 
Therefore, we shall also assume (3) without attempting any 
further investigation of its validity in this paper. 

The CPS labor force classifications for each month of 
the first quarter of the year are the outcome variables in our 
analysis. Let A, B, and C denote the observed classifica- 
tions and let X, Y, and Z denote the (unobserved) true 
classifications for January, February, and March, respec- 
tively. Let G denote some grouping (or stratification) 
variable to be defined later in the analysis. Under these 
assumptions, we can write the probability for classifying a 
CPS sample member in cell (g, a, b, c) of the GABC table 
as follows: 
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Te, a,b,c = x MMe ig Maig,x Myix,g “bly, g MzIg,y “cl, 2" (4) 
X,Y,Z 


Extensions to more than one grouping variable are straight- 
forward. 

Under multinomial sampling, the likelihood function for 
the GABC table is 


Pr(GABC)=k |] 1,%,, (5) 


g,a,b,¢ 

where & is the multinomial constant and II denotes the 
product of the terms over the subscripts g, a, b, and c. 
Under the assumptions made previously, the model para- 
meters are estimable using maximum likelihood estimation 
methods. Van de Pol and de Leeuw (1986) provides the 
formula for applying the E-M algorithm to estimate the 
parameters of this model and describes the conditions for 
their estimability. The 0EM software, applied to the CPS 
data sets in the next section, implements these methods. 


3. APPLICATION TO THE CPS 


3.1 Notation 


Part of our evaluation of the MLCA approach will 
compare the MLCA estimates of classification error with 
estimates derived from the analysis of interview-reinterview 
data. Using the notation in the previous section, let A and 
A’ denote the labor force classification for the original and 
reinterview, respectively, and define 2, = Pr(A = a)and 
1, = Pr(A' =a’). Let AA ‘denote the observed interview- 
reinterview K x K cross-classification table and let 244 !* 

denote the K x K matrix of cell probabilities, Pr(A =a, 
‘'A' =a \X =x). If we. assume: that 2,.)=Pr(A =a, 
A' =a'|\X =x) = nm, referred to in the literature as the 


‘a\x. 


assumption of parallel measures (Bohrnstedt 1983), then 


4 1X =F fiche (greens) We (6) 


where (24!*)? 
tional probabilities, 2 

Let m* denote the K-vector of true classification pro- 
babilities. Then 


denotes the transpose of vector of condi- 
AIX 


ne = nts |X 4X (7) 


i.e., the probability of the observed interview-reinterview 
classification table, 24’, is equal to the product of the 
matrix of conditional response probabilities, 744'*, and 
the vector of true classification probabilities, n*, 

As described in the previous section, the MLCA of the 
CPS longitudinal data will provide maximum likelihood 
estimates of 24'* and 2%, allowing the estimation of 244 
via (6) and (7). We can estimate the test-retest reliability, 
R, for any labor force category by applying the usual esti- 
mation methods (see, for example, Bohrnstedt 1983) to this 
estimate of 244. For our analysis, we compute the index 


of inconsistency, J = 1 - R, which is the traditional reli- 
ability measure for CPS labor force data (see U.S. Bureau 
of the Census 1985). Let J, denote the index of inconsis- 
tency for category A =a. Then an estimator of J, is 


gdr 
2, (1 - f,) (8) 


where gadr is the gross difference rate defined by 
gdr , =2 ye 1 (9) 


ata’ 


and where %, and 7, ,, denote latent class estimates of 1, 
and 1, ,,, respectively. 

U.S. Bureau of the Census (1985, 88-91) provides the 
formulas for standard errors as well as an aggregate 
measure of inconsistency for all K categories combined, 
referred to as the aggregate index of inconsistency, J,,.. 
The aggregate index is a question-level measure of unrelia- 
bility equal to 1 — x (Hess, Singer and Bushery 2000) where 
« is Cohen’s kappa reliability measure (Cohen 1960) and is 
a weighted average of the category-level indexes. 

Finally, given an estimate of 2* we can estimate the K- 
vector of measurement biases, denoted by B ,, associated 
with the K categories of A using the identity 


6 i= 7k: (10) 


3.2 Assessing the Validity of the MLCA 
Methodology 


The primary objective of this paper is to assess the 
validity of the MLCA approach. Previous research in the 
measurement of CPS classification error has not fully 
addressed the validity of the estimation approaches used 
(Meyers 1988). We hope to determine whether the MLCA 
approach is informative and useful for studying classifica- 
tion error in the CPS. In particular, we aim to determine 
whether the model estimates of error probabilities, 24'*, 
reflect the actual levels of error in the CPS labor force 
classifications. Unfortunately, for the reasons mentioned 
previously, no generally accepted gold standard exists for 
assessing the accuracy of the CPS (see, for example, 
Sinclair and Gastwirth 1996, 1998, Biemer and Forsman 
1992, and Schreiner 1980). Consequently, estimating the 
bias of MLCA estimates is not possible. 

In what follows, we will investigate the validity of the 
MLCA estimates of CPS classification error using five 
criteria: 


1. Model diagnostics. A necessary condition for model 
validity is that the model is plausible (i.e., the assump- 
tions are reasonable and are consistent with reality) 
and fits the data adequately. We use the traditional 
chi-square goodness of fit criteria and other diagnostic 
measures of model fit to assess the adequacy of the 
model specification and the degree to which the data 
are consistent with the model. 
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2. Model Goodness of Fit Across Years of CPS. An 
often-used technique for model validation is to assess 
the fit of the model for data that are independent of the 
data used for model building (see, for example, 
Kleinbaum, Kupper and Muller 1988, 330). This 
method is useful for avoiding model over-parameteri- 
zation and data-driven (rather than theory-driven) 
model selection. In the present study, fitting the same 
model to data for each year separately is a form of this 
independent model verification technique. Model 
agreement across the years would tend to support the 
validity of the model structure. This method has a 
difficulty in the present application. After 1993, the 
CPS paper and pencil questionnaire was redesigned 
for Computer Assisted Personal Interview (CAPI) 
administration, so the magnitudes of the response 
errors may have changed after 1993. However, if the 
primary sources of response error in the CPS have not 
changed with the redesign, a model structure that 
adequately describes the error for 1993 should also 
describe the error for 1995 and 1996. 


3. Agreement of the MLCA Estimates and the Hui- 
Water Test-Retest Estimates of Response Proba- 
bilities. The Hui-Walter (H-W) method (Hui and 
Walter 1980) for estimating CPS response proba- 
bilities uses unreconciled reinterview data (Sinclair 
and Gastwirth 1996; 1998). Although the MLCA and 
H-W methods both use latent class models, the model 
assumptions are very different. For example, the H-W 
method does not require the Markov assumption for 
model identifiability. Further, in this research, the 
data inputs to the H-W method are independent of 
those used for the MLCA method. Close agreement 
between the two sets of estimates supports the validity 
of both methods, while poor agreement suggests that 
at least one of the approaches is not valid. Strong 
agreement between the MLCA and H-W estimates 
also lend some assurance that the MLCA estimates of 
response probabilities are relatively robust to possible 
violations of the Markov assumption. 


4. Agreement of Model and Test-Retest Estimates of 
the Index of Inconsistency. This criterion is similar 
to Criterion 3 because it compares estimates derived 
from MLCA with estimates based upon unreconciled 
reinterview data. However, this analysis does not rely 
on the validity of the Hui-Walter estimation methodol- 
ogy to assess MLCA estimation validity. Instead we 
use the MLCA estimates of classification error to 
compute estimates of the index of inconsistency using 
(7) to (9). We compare these estimates of reliability 
directly to the estimates of reliability from the CPS 
Reinterview Program, obtained from unreconciled 
reinterview data. Good agreement between the 
Reinterview and MLCA estimates supports the 
validity of both methods, while poor agreement 
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suggests that at least one of the approaches is not 
valid. 


5. Plausibility of Patterns of Classification Error. 
Finally, the plausibility (or face validity) of the 
response probability estimates can also provide a test 
of validity. For example, it seems implausible that 
proxy responses to labor force questions should be 
more accurate than self-responses. Other patterns of 
classification error can also be reviewed and evaluated 
for plausibility. To the extent that the model estimates 
seem plausible, the face validity of the estimates is 
supported. 


In the next section, we discuss our MLCA modeling results 
in the context of these criteria for validity. We begin with 
a description of the CPS data sets and the results of the 
model selection process. 


3.3. The CPS Data Sets 


In 1994, in conjunction with the implementation of 
computer assisted personal interviewing (CAPD), the CPS 
underwent a major redesign and a restructuring of the 
questions used to determine labor force status. Rothgeb 
(1994) provides a description of the CPS redesign. As a 
result of these improvements, we expect to see a difference 
(specifically a reduction) in classification error for the post- 
1994 CPS relative to 1993. Although not a primary objec- 
tive of this research, we compared the error in the CPS 
before and after the redesign. We tested the MLCA 
approach for three years of the CPS — 1993, 1995, and 1996 
— because the CPS unreconciled reinterview data were 
readily available for these time periods. 

The CPS households are interviewed for four consecu- 
tive months, drop out of the survey for eight months, and 
then re-enter to be interviewed for a second series of four 
consecutive months. MLCA requires at least three consecu- 
tive interviews for identifiability of the model parameters. 
We had a choice of data sets which included all persons 
interviewed in three or four consecutive months of the CPS. 
Since using four months of data would reduce the sample 
size for the analysis by half, we chose to focus the analysis 
on three consecutive months — January, February, and 
March — for all three years of data. Nonresponse cases and 
cases where the whole household changed in one or more 
of the three months were excluded from the analysis. 

The simplest MLCA model specifies that the response 
probabilities, 7,,,, 7 bly? and z,,,, and the transition proba- 
bilities, 1,,,, 7, are the same for all persons in the target 
population (referred to as homogeneity). However, our pre- 
liminary analysis (Biemer, Bushery and Flanagan 1997) 
indicated that response and transition probabilities where 
not homogeneous. To account for this heterogeneity, we 
explored a number of covariates and stratification variables 
for inclusion in the models, including: gender, education, 
mode of interview, proxy/self-response, and race. Of the 
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those considered, a variable derived from the CPS proxy/ 
self response indicator best accounted for population 
heterogeneity. This variable, denoted by P, is defined as 
follows: 


1 ifall three interviews are conducted 
by self-response (SELF) 

2 if two of the three interviews are conducted 
by self-response (MOSTLY SELF) 

3 if two of the three interviews are conducted 
by proxy response (MOSTLY PROXY) 

4 if all three interviews are conducted 
by proxy response (PROXY) 


Note, we now use P to represent the grouping variable, in 
place of G, which we used in section 2. Based upon 
previous research (for example, O’ Muircheartaigh 1991), 
we expect that the Self group (P=1) to have less 
classification error than the Proxy group (P=4). We test 
this hypothesis as part of the estimate plausibility criterion 
(criterion 4 above). 

The sample sizes for the three data sets used in our 
analysis are 


1993: 45,291 persons 
1995: 49,347 persons 
1996: 41,751 persons 


For 1993, approximately one-third of the sample is in the 
Self group, approximately one- fourth in the Proxy group, 
and the remaining sample members are distributed approxi- 
mately equally between the Mostly Self and Mostly Proxy 
groups. For 1995 and 1996, slightly more sample members 
(one-third rather than one-fourth) are in the Proxy group. 


3.4 Fitting the MLCA Models 


To fit an MLCA model with a single grouping variable, 
P, the input data set was a 4x3x3x3 table of cell counts 
defined by the cross-classification of PxAxBxC, where A, 
B, and C are the labor force classifications for January, 
February, and March, respectively. 

The 0EM software and other software packages for 
fitting MLCA models assume simple random sampling, so 
the complex survey design of the CPS cannot be modeled 
exactly. It is possible to account for the unequal probability 
sampling structure of the CPS through the use of weighted 
and rescaled cell counts rather than the raw cell totals 
(Clogg and Eliason 1985). However, using unweighted data 
for the MLCA analysis affords two important advantages. 
First, we can compare the MLCA estimates with estimates 
from the previously cited studies on CPS classification 
error, all of which used unweighted data. Second, the CPS 
reinterview data used to assess Criteria 3 and 4 are 
unweighted and weights are not available. Consequently, 
at least part of the analysis requires unweighted data; using 
weighted data for the other criteria could produce spurious 
inconsistencies in the results. 


To investigate the validity of inferences to the total 
population using unweighted analysis, we estimated classi- 
fication errors from both weighted and unweighted data and 
observed that the classification error estimates expressed as 
proportions were virtually identical, differing only at the 
third decimal place. Thus, the results we report below using 
unweighted cell counts are appropriate for inference beyond 
the CPS sample to the total population. 

Another consideration in using unweighted analysis is 
the estimation of standard errors. Since they are computed 
using simple random sampling assumptions, the 0EM 
standard error estimates may be understated as a result of 
ignoring the clustering effects in the CPS sample. To appro- 
ximately account for this, we can multiply the 07EM 
variances by a design effect computed from the CPS labor 
force estimates. U.S. Bureau of the Census (2000, 14-9) 
indicates that the design effects for the CPS labor force 
estimates do not exceed 1.3 and thus multiplying the 2EM 
standard errors by (1.3) should inflate the standard errors 
sufficiently to account for clustering. An equivalent 
approach is to use a 3 percent rather than a 5 percent level 
of significance in declaring the difference between two 
estimates to be statistically significant. This latter strategy 
will be employed in the forthcoming analysis as appro- 
priate. We believe this produces a conservative test since 
the CPS design effect reflects the increase in variance due 
to both sample clustering and unequal weighting, while 
only clustering effects are present our unweighted 
estimates. 

Table 1 shows the results of fitting a sequence of 
increasingly complex MLCA models for each of the three 
data sets. The Base Model is the simplest MLCA model 
and specifies that transition probabilities and response 
probabilities are homogeneous (i.e., do not differ by group, 
P) and stationary (i.e., are the same for all three months). 
This model may be written as 


TL, a,b,c sae 1, Nig ies ae (11) 


which is obtained from (4) by imposing the constraints 


Mei yp = Rytzp = Byis (12) 


z|yp yixp 
and 


Na \xp 7; Ns iyp e Neizp it Ta | x (13) 


for all p. 
For Model 1 we relax constraint (12) to 


Rayo Py p for p =1,...,4 (14) 


and thus allow transitions from January to February and 
February to March to vary by Self/Proxy Group, P. For 
Model 2, we further relax constraint (12) to 


=H), and 1), = Ty (15) 


Tt 
y|xp 
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Table 1 


Model Diagnostics for Alternative MLCA Models by Year 
——— ee es 


1993 Data df npar! i p-value BIC d 
Base Model: Homogeneous and stationary transitions and 90 17 645 0 -320 0.048 
response probabilities 

Model 1: Nonhomogeneous transitions 84 23 632 0 -269 0.047 
Model 2: Non-stationary transitions 66 41 99 0.006 -609 0.01 
Model 3: Nonhomogeneous and non-stationary transitions 42 65 64 0.016 -386 0.01 
Model 4: Nonhomogeneous and non-stationary transitions and 24 83 23 0.501 -234 0 
nonhomogeneous response probabilities 

1995 Data af npar' i p-value BIC d 
Base Model: Homogeneous and stationary transitions and 90 17 697 0 -275 0.044 
response probabilities 

Model 1: Nonhomogeneous transitions 84 px) 668 0 -240 0.043 
Model 2: Non-stationary transitions 66 41 146 0 -567 0.01 
Model 3: Nonhomogeneous and non-stationary transitions 42 65 82 0) -372 0.01 
Model 4: Nonhomogeneous and non-stationary transitions and 24 83 25 0.41 -234 0 
nonhomogeneous response probabilities 

1996 Data df npar’ i? p-value BIC d 
Base Model: Homogeneous and stationary transitions and 90 17 632 0 -325 0.045 
response probabilities 

Model 1: Nonhomogeneous transitions 84 Dp 585 0 -308 0.044 
Model 2: Non-stationary transitions 66 41 159 0 -543 0.01 
Model 3: Nonhomogeneous and non-stationary transitions 42 65 82.6 0 -364 0.01 
Model 4: Nonhomogeneous and non-stationary transitions and 24 83 3933 0.026 -216 0 


nonhomogeneous response probabilities 


= SS SS SS 12 sass a aaaai«S<S_ SS _  _ 


Note that “npar” refers to the number of parameters in the model 


for all p. Model 3 relaxes both the homogeneity and 

stationarity constraints for transition probabilities so that 

™,,., #7_,,_. Thus this model allows transition probabilities 
yixp_ | z\|yp 

to vary by group and by month. However, response proba- 

bilities are still constrained to be equal across groups and 

months. 

Model 4 is the most general, identifiable model we 
considered. Model 4 allows the January-February and 
February-March transition probabilities to vary indepen- 
dently across the four proxy/self groups. This model further 
species that the response probabilities are the same for 
January, February, and March, but may vary across the four 
proxy/self groups. We obtained this model from Model 3 by 
relaxing the constraints specifying homogeneous response 
probabilities; i.e., by relaxing constraint (13) to 


(16) 


for all p. Under these constraints, (4) can be written as 


Ng | xp . Nolyp a Ne zp 


= 3 
TL, a,b,c i y MTs Ip Ty trp Mei py Mal p, x) : 
LY, Z 


In Table 1, we show the basic fit statistics for all five 
models for all three years. Column 4 of the table provides L *, 


the usual likelihood ratio chi-square statistic (see Agresti 
1990, 48), and column 5 the corresponding p-value. A 
p-value of 0.05 or greater is the usual criterion for adequate 
model fit. However, due to the large sample sizes in our 
analysis, requiring a p-value this large could result in model 
over fitting. We consider a p-value as small as 0.01 to be 
acceptable. The BIC measure in the table is defined as 


BIC = L*-(log N)df 


where N is the total sample size and df is the degrees of 
freedom for the model. The BIC essentially summarizes the 
tradeoff between model fit (L2) and model parsimony (d/). 
Since small values of the BIC are favorable, we will regard 
the model with the smallest BIC as best with respect to 
goodness of fit and parsimony. Liu and Dayton (1997) 
discuss this approach for latent class models. 

Finally, the dissimilarity index (d) is the proportion of 
observations that would have to change cells for the model 
to fit perfectly. As rule of thumb, models having d < 0.05 
(i.e., 5 percent model error) are considered to fit the data 
well (Vermunt 1997). 

For each year of data, Model 4 is the only model to 
provide an acceptable fit when the p-value criterion is 
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considered. Model 4 is also plausible from a response 
theory perspective since it postulates that classification 
error varies by self/proxy group. This, as we have said, is 
consistent with the survey methods literature (see for 
example, O’Muircheartaigh 1991 and Moore 1988). The 
dissimilarity index, d, for the model is 0.3 percent, which 
indicates a very good model fit. Thus, we use Model 4 to 
generate the estimates of labor force classification error. 


3.5 Estimation of Classification Error 


Table 2 shows estimates of the response probabilities 
from Model 4 for Employed, Unemployed, and Not in the 
Labor Force. For the true Employed and those truly Not in 
the Labor Force, the probability of a correct response is 
quite high: at least 98 percent for Employed and 97 percent 
for Not in the Labor Force. However, for the true 
Unemployed, the probability of a correct response varies 
across years and groups from approximately 68 percent to 
approximately 86 percent. As expected, the Self group has 
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the highest probability of a correct response (statistically 
significant for 1993 and 1996). The Mostly Self group also 
displays a tendency for a higher probability of a correct 
response than the Mostly Proxy group, but the difference is 
not statistically significant. 

A surprising result from Table 2 is the direction of the 
difference in reporting accuracy for the true unemployed 
across the three years. Recall that the CPS interview 
questionnaire was redesigned in 1994 to increase reporting 
accuracy. However, these results suggest that reporting 
accuracy is higher for 1993 (the year prior to the major 
redesign) than for 1995 and 1996 (the years following the 
redesign). In 1993, the probability of a correct response 
was 81.8 percent, compared with 76.1 percent and 74.4 
percent for 1995 and 1996, respectively (p-value <0.001)). 
This effect may be a consequence the redesign or may 
reflect actual changes in the population, or both. Investi- 
gations are currently underway at the Census Bureau to 
understand the causes underlying these results. 


Table 2 
Estimated Labor Force Classification Probabilities by Group and Year 
(Standard Errors are in parentheses) 


Observed True Employed 

Group 1993 1995 1996 

Emp Self 98.90 98.98 99.03 
(0.12) (0.12) (0.13) 

Mostly 98.91 98.61 98.86 
Self (0.18) (0.21) (0.22) 

Mostly 98.62 98.56 98.76 
Proxy (0.18) (0.22) (0.20) 

Proxy 98.66 98.63 98.66 
(0.16) (0.13) (0.15) 

Total 98.77 98.73 98.83 
(0.11) (0.11) (0.11) 

Unemp __ Self 0.33 0.39 0.41 
(0.08) (0.08) (0.09) 

Mostly 0.28 0.50 0.28 
Self (0.12) (0.13) (0.15) 

Mostly 0.37 0.54 0.37 
Proxy (0.11) (0.15) (0.13) 

Proxy 0.34 0.55 0.39 
(0.10) (0.09) (0.09) 

Total 0.34 0.49 0.37 
(0.11) (0.11) (0.11) 

NLF Self 0.77 0.63 0.55 
(0.10) (0.08) (0.09) 

Mostly 0.81 0.89 0.86 
Self (0.13) (0.15) (0.16) 

Mostly 1.01 0.90 0.87 
Proxy (0.14) (0.15) (0.15) 

Proxy 0.10 0.82 0.95 
(0.13) (0.10) (0.12) 

Total 0.89 0.78 0.79 
(0.11) (0.11) (0.11) 


True Unemployed True NLF 
1993 1995 1996 1993 1995 1996 
4.24 5.66 5.78 1.05 0.67 0.92 
(0.96) (1.28) (1.52) (0.13) (0.10) (0.13) 
8.11 7.66 12.20 73 1.59 0.93 
(1.89) (2.50) (2.96) (0.27) (0.30) (0.24) 
PISS S6 a 0518 1.68 0.95 1.50 
(C1E9G) Pen(2:977) (2273) a O2257) eae (O22) ae (0-30) 
6.60 Uy 8.30 1.56 1.50 1.30 
(1:43) (1.44) (1.62) ~ (120) » (0:17) (nN8) 
7.06 7.86 8.57 1.4] Labi Nels 
(0.70) (0.90) (1.00) (0.11) (11) (011) 
85.92). 5805255 79.66 0.48 0.35 0.40 
(1.62) (2.04) (2.44) (0.11) (0.10) (0.12) 
S208 5) or O0 1.03 0.91 0.79 
(2.48) (3:40) G71), 4:23) (O21) (0228) 
76.54 68.09 72.74 0.81 0.90 53 
(2°55) (6:37)! 9 G70), (0:22) (O28) (O26) 
SOOO 7al2 eos 0.95 0.94 1.18 
(2.19) (2.18) (2.49) (0.19) (0.17) (0.20) 
81.81 76.09 74.42 0.75 0.69 0.87 
(0:90) 2 1) 2 Osa es ONL) OM 
9.84 13.82 14.56 98.47 98.98 98.68 
(E39) Pas 6) ee (2.1) (O17) ea (O514) ae (OLS) 
9.21 20:63 7 914-200 097-245 ya97 0m 98-28 
(1.83) (2.94) (2.94) (0.35) (0.36) (0.37) 
1227 1905 ST OS oie) 2a 9 OulS en 90:96 
(1.97) (2.58) (3.16) (0.35) (0.38) (0.40) 
13:31 Se O07 97:49 OF -6N 9/252 
GES3) E86) a 222) OD) O24) O27) 
LVS) 1G 04 7200) 897-845 98:20 98:00 
(O90) G21) eT) ONL) ee (OA ee (ORT) 
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Table 3 
Comparison of MLCA Estimates with Prior Published Estimates 
Classification MLCA Chua & Fuller Poterba & CRS 
(1982 data) Summers Reconciled 
True Observed (1981 data) Reinterview 
(1977-1982) 
Employed Emp 98.77 (1993) 98.66 (month 1) 97.74 98.78 
98.73 (1995) 98.65 (month 2) 
98.73 (1996) 
Unemp 0.34 (1993) 0.32 (month 1) 0.54 0.19 
0.49 (1995) 0.34 (month 2) 
0.37 (1996) 
NLF 0.89 (1993) 1.02 (month 1) ee 1.03 
0.78 (1995) 1.01 (month 2) 
0.79 (1996) 
Unemp Emp 7.06 (1993) 3.52 (month 1) 3.78 1.91 
7.86 (1995) 3.51 (month 2) 
8.57 (1996) 
Unemp 81.81 (1993) 88.27 (month 1) 84.76 88.57 
76.09 (1995) 88.23 (month 2) 
74.42 (1996) 
NLF 11.13 (1993) 8.21 (month 1) 11.46 9.53 
16.04 (1995) 8.16 (month 2) 
17.00 (1996) 
NLF Emp 1.41 (1993) 1.60 (month 1) 1.16 0.5 
1.11 (1995) 1.61 (month 2) 
1.13 (1996) 
Unemp 0.75 (1993) 1.19 (month 1) 0.64 0.29 
0.69 (1995) 1.24 (month 2) 
0.87 (1996) 
NLF 97.84 (1993) 97.21 (month 1) 98.2 99.21 


98.20 (1995) 97.15 (month 2) 


98.00 (1996) 


The table indicates that misclassification of the 
unemployed as NLF is a bigger problem than misclassifi- 
cation as Employed. Averaging over all three years, 
approximately two thirds of the error in classifying the 
unemployed is misclassification as NLF. But the rates of 
both types of error are high. 

Next, we compare our estimates of the CPS classification 
probabilities with similar estimates from the literature. In 
Table 3, the MLCA estimates for each of the three years are 
compared with estimates from Chua and Fuller (1987), 
Poterba and Summers (1995), and the CPS reconciled 
reinterview program. Again, the latter three sets of esti- 
mates rely on reinterview data while the MLCA estimates 
are produced directly from the CPS interview data. In 
general, the relative magnitude of the MLCA estimates 
across the labor force categories agrees with the previous 
estimates. The greatest differences occur for the true 
unemployed population. For this group, the estimates of 
response accuracy from the literature are three to seven 
percentage points higher than corresponding MLCA esti- 
mates for 1993, which is the time period that most closely 
corresponds to the comparison estimates. 


One explanation for this difference is that the compa- 
rison estimates are biased upward as a result of correlations 
between the errors in interview and reinterview. Another 
explanation is that the MLCA estimates are biased 
downward as a result of the failure of the Markov 
assumption to hold. We suspect that both explanations may 
be true to some extent. However, the next section provides 
some evidence that failure of the Markov assumption likely 
has a small effect on estimates of classification error. 


3.6 Robustness of MLCA to Non-Markov Labor 
Force Transitions 


A number of authors have investigated the effects of 
current and previous employment status on future 
employment status (see, for example, Akerlof and Main 
1980; Heckman and Borjas 1980; Lynch 1989, and Corak 
1993). Heckman and Borjas show that examination of this 
issue is quite difficult due to selection biases, response 
error, and unobserved heterogeneity. These confounding 
influences may account for the inconsistent findings in the 
literature. For example, using data from the CPS, Akerlof 
and Main (1980) provide evidence that the probability of 
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future unemployment depends upon the number of previous 
unemployment spells experienced as well as the duration of 
those spells. However, in a study of male high school 
graduates, Heckman and Borjas (1980) found “no evidence 
that previous occurrences of unemployment or their 
duration affect future labor market behavior once we 
control for sample selection bias and heterogeneity bias.” 
The results from the literature are also inconsistent and 
ambiguous regarding the extent to which the Markov 
assumption expressed in (2) may be violated for the CPS 
and other labor market surveys. Nevertheless, in this 
section, we attempt to provide at least a partial answer to 
question of how non-Markov labor force transitions affect 
MLCA estimates of classification error. 

To investigate the effect of violations of the Markov 
assumption in (2) for the present application, we conducted 
a limited simulation study. To focus the investigation while 
simplifying the simulation framework, we considered latent 
structures involving only two classes or states at each time 
point: unemployed, denoted by X, Y, or Z = 1, and other 
(i.e., employed or not in the labor force), denoted by_X, Y, 
or Z = 2 with analogous definitions for the observed states 
A,B, and C. To create a population for the simulation, the 
latent probabilities 7,, Ty) x9 and Te) and the response 
probabilities Naix = Mp \y = Te, Were specified to be consistent 
with the combined 1993, 1995, and 1996 data sets. 

We then defined two parameters, 1, and i, to be varied 
in the simulation, where 

™ 
M pM at WH) at (17) 


M2 1[x=1,y= 1 
and 


7 
Peal a Dayae 
ge (18) 
Me =1|x=1,y=2 


Thus, A, is the probability of being “unemployed” in 
March, given “unemployed” in February and “other” in 
January over the probability of being “unemployed” in 
March given “unemployed” in the two previous months. 
Consistent with the findings of Akerlof and Main (1980) 
who showed that the likelihood of remaining unemployed 
increases as the number of unemployment spells increases, 
we assume that 0<,<1. Similarly, 4, is the probability of 
being “unemployed” in March, given “other” in the two 
previous months, over the probability of being 
“unemployed”, given “other” in February and 
“unemployed” in January. Again, by Akerlof and Main, we 
assume 0<A,<1. Note that when 4, =, = 1, unemploy- 
ment transitions from February to March are Markov. 
The simulated data were generated to be completely 
consistent with a MLCA model having non-stationary 
transition probabilities when 1, =2,= 1. We simulated 
failure of the Markov assumption by varying 4, and i, 
between 0 and 1. To be consistent with the 1993-1996 data, 
we fixed the probability of a correct “unemployed” 
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response, 7, _1),-4» at 0.80 and the probability of a correct 
“other” response, T,-2)x=2°at 0.99 in all simulations. In 
addition, the denominators of 1, and 2, were fixed to their 
values as determined from the combined 1993-1996 data 
while the numerators were computed from (17) and (18) 
using the values of 4, and 1, specified in each simulation 
run. 

Table 4 summarizes the results of the simulation for 
i, =A, =A where A is varied from 0.2 to 1.0 in steps of 0.2. 
Note that for 1, = r, = 1.0, which corresponds to a Markov 
model, the estimated probabilities of correct response are 
exactly as specified. For smaller values of 4, and X,, the 
estimates become negatively biased and are most biased for 
the lowest value considered, 0.2. Nevertheless, the absolute 
biases due to non-Markov transitions probabilities are never 
more than 3 percentage points. The results in Table 4 are 
consistent with Bushery and Kindelberger (1999), who used 
a somewhat different approach to illustrate the same 
robustness property of the MLCA models for CPS data. 
Both studies suggest that failure of the Markov assumption 
to hold does not appear to be an important source of bias in 
estimating CPS classification error probabilities. 


Table 4 
Estimates of Correct Classification Under 
Non-Markov Transitions 
(Cell entries are percentages) 


M=h=A 

Pr (Correct) h=1.0 
1=0.2 2=04 2=06 2=08 Gyayivy 

Ep Gunemp\Luve 17.6.1 IS1> SISIN 9S 80 

unemp”) = es TPES | 

Pr (“other”! true 98.6 98.7 988 98.9 99 

other”) = 1, =21=2 


4. COMPARING THE MLCA AND 
UNRECONCILED REINTERVIEW 
ESTIMATES 


4.1 Hui-Walter Estimation 


An alternative set of response probability estimates can 
be obtained from the CPS reinterview data using a type of 
latent class model first proposed by Hui and Walter (1980). 
Using the notation introduced above, let X denote the true 
labor force classification for some time point and let A and 
A’ denote the interview and reinterview classifications, 
respectively. Let G denote a grouping variable defined as 
in (4). Consider the likelihood of the groupx interviewx 
reinterview table denoted by GAA’. Denote by Teaa' the 
probability of classifying an individual belonging to group 
g into cell (a, a’) of the table. The model for Tegaa' 
proposed by Hui and Walter is 


Mea’ » MyM |g Max Ma’|x° (19) 


NOMINATIONS SOUGHT 
FOR THE WAKSBERG 
INVITED PAPER SERIES 


Survey Methodology has established an annual invited 
paper series in honor of Joseph Waksberg, who has made 
many important contributions to survey methodology. 
Each year, as part of the Waksberg Invited Paper Series, a 
prominent survey researcher will be chosen to author a 
paper that will review the development and current state of 
a significant topic within the field of survey methodology, 
and will reflect the mixture of theory and practice that 
characterizes Waksberg’s work. The author will receive a 
cash award made possible by a grant from Westat, in 
recognition of Joe Waksberg’s contributions during his 
many years of association with Westat. The grant is 
administered financially by the American Statistical 
Association. 


The author will be selected by a four-person committee 
appointed by Survey Methodology and the American 
Statistical Association. Nominations of individuals to be 
considered as authors should be sent to the chair of the 
committee, Graham Kalton, at Westat, 1650 Research 
Blvd., Rockville, MD 20850, by e-mail to 

kaltong1 @westat.com, or by fax to 301-294-2034. 
Nominations for the author of the 2002 Waksberg paper 
must be received by March 16, 2001. 
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In this model, the parallel] measures assumption for the 
interview and reinterview responses is relaxed and response 
probabilities for the two measures, viz. T,), and 1,",,, are 
estimated separately. The ICE assumption is made as a 
condition of identifiability. It is further assumed that 
T,),and T,'|,d0 not depend upon the group variable, G, 
while the prevalence of employed, unemployed, and NLF, 
1.€. To still depends upon G. 

Sinclair and Gastwirth’s (1996) analysis of CPS labor 
force classification error used Sex as the grouping variable 
and our analysis uses this grouping variable as well. 
Sinclair and Gastwirth confined their analysis to white 
males and females and two labor force categories: NLF and 
In the Labor Force. The latter category is the sum of our 
Employed and Unemployed categories. In our analysis, we 
consider sample members of all races and analyze the three 
category labor force classification used in the MLCA. 
Thus, the H-W analysis estimates 16 parameters for each 
year, which equals the number of degrees of freedom 
available from the GxA xA ‘table, leaving no degrees of 
freedom to test model fit. 

The 0EM software was used to fit the H-W model to the 
interview and unreconciled reinterview data from three time 
periods that coincide with the three in our MLCA: pre- 
1994, 1995, and 1996. We attempted to restrict the analysis 
to only the first quarter of these time periods. Unfortuna- 
tely due the small sample sizes, the estimates were quite 
unstable. Thus, it was necessary to use the reinterview data 
from all four quarters of these time periods. The pre-1994 
data were collected from 1985 through 1988 using the 
unreconciled reinterview sample. 

The results of this comparison of MLCA and H-W 
estimates are summarized in Table 5. The MLCA estimates 
are the same as those in the rows of Table 2 labeled “Total.” 
The H-W estimates are the classification probabilities 
associated with the original interview, i.e., measure A in 
(19). The table shows the comparison for all three years. 
Since the largest error rate in the MLCA occurred for the 
Unemployed, this category is of particular interest in the 
MLCA/H-W comparison. 

Overall, the two sets of estimates show fairly good 
agreement. The years 1995 and 1996 exhibit no statistically 
significant differences (at the 5 percent level) between the 
MLCA and H-W estimates for the unemployed population. 
The pre-1994 estimates display significant differences; 
however, they may be explained by the fact that the pre- 
1994 reinterview data were from 1985 through 1988, rather 
than 1993. These differences will be explored further in the 
next section. 


4.2 Comparison of Indexes of Inconsistency 


As described in section 3.1, we compute estimates of the 
index of inconsistency for each time period using the 
MLCA model-based estimates of the response probabilities. 
Essentially, we estimate the expected interview-reinterview 
cross-classification table from the MLCA response 
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probability estimates and then apply the formula for the 
index to this table as though the table were observed. A 
second expected interview-reinterview classification table 
be can estimated using the H-W response probability 
estimates. We then compared these two sets of estimates to 
the estimate of the index computed directly from the CPS 
reinterview data using traditional methods (U.S. Bureau of 
the Census 1985). Agreement of all the three estimates 
agree supports the validity of the three methods. 


Table 5 
Comparison of MLCA and H-W Model Estimates of CPS 
Response Probabilities by Year 
(Standards Errors are in Parentheses) 


Classification 1993 1995 1996 

True Observed H-W MLCA H-W MLCA H-W MLCA 
Emp Emp 993 98.8 995 987 996 98.8 
(O33) OA) (OF) COD) HOS) 021) 

Unemp 0.0 0.3 0.0 0.5 0.4 0.4 

(0.0) (0.1) (@/a) (0.1) (1) (0.1) 

NLF 0.7 0.9 0.5 0.8 0.0 0.8 

O33 OD) 107)" Ol)" ) alae O 1) 

Unemp Emp 11.1 TI 11.5 7:9 4.6 8.6 


(CEO) (O87) (2:3) en (O09) (5:2) ae EO) 


Unemp Aes eS lES ee. O79 eT Only O7:65e a a4 

QT) GED KG) CES) e (LIS) aes) 

NLF 14.7 LONE 20.6 16:0 27-9 20 

(239) Be (O:9) ee (6S) ae (22) © :3) ee (122) 

NLF Emp 2.0 1.4 WS) 1.1 2.6 1.1 
(O05) Fs (OSD) lS) (O31) aS) ae (O51) 

Unemp 12 0.8 0.5, 0.7 0.0 0.9 

(0.3) (0.1) (06) (01) (n/a) (0.1) 
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Table 6 shows the three methods estimates the index of 
inconsistency for all three time periods. As before, the 
Unemployed category is of particular interest because of its 
large error rate. Standard errors are not available for the 
MLCA or the H-W estimates of the index so formal tests of 
hypothesis are not possible. However, standard errors for 
the traditional estimates are provided which can be used as 
rough approximations of the standard errors for the H-W 
estimates. 

Overall, both the general patterns of the MLCA 
estimates and the magnitudes of the MLCA estimates 
generally agree quite well with the H-W and traditional 
estimates for all three years. However, for the NLF 
category in 1995 and 1996, the traditional estimates of J are 
somewhat larger than either of the latent class model 
estimates. Further analysis suggests that this difference is 
due to a bias in the traditional estimation approach resulting 
from the failure of the parallel measures assumption. 

U.S. Bureau of the Census (1985) shows that if the 
interview and reinterview processes have different reliabi- 
lities, then the traditional estimate of the index will be 
biased. For example, if the reliability of the reinterview 
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data is lower than the reliability of the interview data, the 
traditional test-retest reliability estimator will understate the 
actual reliability of the CPS data; i.e., the CPS index of 
inconsistency will be too large. 


Table 6 
Comparison of MLCA, H-W, and Traditional Estimates of the 
Index of Inconsistency by Year and Labor Force Classification 


Method of Labor Force Classification Aggregate 
pes Employed Unemployed Not in Labor eas 
Force 
1993 
Traditional 8.16 33.49 9.96 11.05 
estimation (0.24) (1.16) (0.27) (0.26) 
H-W 7-37 34.93 10.07 10.78 
MLCA 6.35 28.04 7.63 8.73 
1995 
Traditional 6.69 36.28 10.80 10.42 
estimation (0.44) (2.85) (0.56) (0.53) 
H-W 6.82 37 8.98 O77 
MLCA 6.06 36.19 TZ 8.72 
1996 
Traditional 5.93 35.97 11.95 10.61 
estimation (0.39) (2.68) (0.56) (0.51) 
H-W 5.67 39.46 7.55 8.56 
MLCA 5.99 37.39 7.76 9.06 


The CPS interview and reinterview will have different 
reliabilities if the error distributions for the two interviews 
are not equal. A test of this is possible by comparing the fit 
of a H-W type model with and without the restriction 
T,\, = Marx: The assumption of equal reliability is rejected 
if ie difference between the likelihood ratio chi-squares for 
the two models exceeds a chi-square with 6 degrees of 
freedom. This test was rejected for 1995 and 1996 at the 10 
percent level of significance. Thus, it appears that the 
difference in the NLF estimates for 1995 and 1996 may be 
due, in part, to bias in the traditional estimates of J. 

Note further that the H-W and MLCA indexes agree 
quite well for 1995 and 1996, although they differ 
somewhat in 1993. However, as noted in the discussion of 
Table 5, the comparisons between the MLCA and H-W 
estimates for this year are confounded by the different time 
periods used to construct the pre-1994 interview- 
reinterview data set. This could account for at least some 
of the discrepancy between the estimates for this year. 


5. SUMMARY AND CONCLUSIONS 


The primary goal of this research was to investigate the 
validity of MLCA estimates of CPS labor force classifi- 
cation error and to determine the efficacy of MLCA as an 
alternative to traditional methods for evaluating CPS data 
quality. We analyzed interview data from the CPS for the 


first quarter of three years — 1993, 1995, and 1996 — and 
conducted an additional analysis of the CPS unreconciled 
reinterview data for approximately the same time periods. 
The reinterview data provided another approach for 
estimating CPS classification error that, when compared 
with the MLCA estimates, helped to address the question of 
the validity of the MLCA approach. 

Five dimensions of MLCA validity were addressed as 
follows: 


1. Model diagnostics. We investigated a wide range of 
MLCA models with grouping variables defined by 
age, race, sex, education, mode of interview, and 
proxy/self response. The most parsimonious and best 
fitting model for all three years included one grouping 
variable defined by the proxy/self variable with four 
categories: all three waves conducted by self response, 
only two waves conducted by some self response, on- 
ly two waves conducted by proxy response, and all 
three waves conducted by proxy response. For this 
class of models, the best model was Model 4 (see 
Table 1) which specified non-homogeneous and non- 
stationary transition probabilities and non-homoge- 
neous response probabilities. This model provided an 
adequate fit to the data for all three years. 


2. Model Goodness of Fit Across Years of CPS. 
Another indicator of model validity is its fit across 
independent samples of the same population. 
Assuming that labor force dynamics and the response 
probability structure for the CPS is stable across the 
span of four years, the same general model should fit 
all three years adequately. Model 4 displays multi- 
year goodness of fit (see Table 1). In addition, other 
grouping variables were tested in the study, yet the 
proxy/self variable model emerged as the best variable 
for all three years. 


3. Agreement Between the Model and Test-Retest 
Estimates of Response Probabilities. Using the 
unreconciled interview-reinterview data from the CPS 
for the time periods pre-1994, 1995, and 1996, we 
applied the H-W method to estimate the response 
probabilities and compared these with the MLCA 
estimates. There was good agreement for 1995 and 
1996, the two years for which the time periods for the 
reinterview data and the CPS data were closely 
matched (see Table 5). For 1993, we observed small 
but significant differences between MLCA estimates 
and the corresponding H-W estimates. These 
differences might be explained by differences in the 
time periods, since the reinterview data predated the 
CPS interview data by some years. 


4. Agreement Between the Model and Test-Retest 
Estimates of Inconsistency. We compared MLCA 
model-based estimates of the index of inconsistency 
with the corresponding direct estimates from the CPS 
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reinterview program. The two sets of estimates agree 
fairly well for all three years, with the exception of the 
NLF category (see Table 6). For 1995 and 1996, the 
differences can be partly explained by the bias in the 
traditional estimator resulting from the failure of the 
parallel measures assumption. The H-W method, 
which does not require the assumption of parallel 
measures, produces estimates of the index that agree 
well with MLCA estimates for 1995 and 1996. For 
1993, the difference between the MLCA and H-W 
estimates may be due to the difference in the time 
periods for the reinterview and the CPS data sets. 


5.  Plausibility of the Patterns of Classification Error. 
The MLCA estimates of misclassification probabi- 
lities appear to be plausible. The estimates across 
proxy/self groups were consistent with prior expecta- 
tions that lower error rates should be observed for self 
respondents than for proxy respondents. In addition, 
the largest error rates were observed for the 
unemployed population and the magnitudes of these 
estimates were consistent with those of previous 
studies — for e.g., Fuller and Chua 1985; Abowd and 
Zellner 1985; Porterba and Summers 1986; and 
Sinclair and Gastwirth 1996 (see Table 3). 


In summary, we found no evidence from these analyses 
to question the validity of the MLCA approach. The method 
performed well in all five validity tests. We therefore 
recommend that the MLCA method be considered as an 
alternative method for evaluating the accuracy of the CPS 
labor force estimates. The strong agreement between the 
MLCA and H-W estimates supports the validity of the H-W 
method as well. We recommend that both methodologies 
be considered in future studies of CPS data quality. 

Although the MLCA approach performed well in our 
tests, we recommend caution in applying the methodology 
in other settings. In our analysis, reinterview data provided 
a means for assessing the validity of the MLCA estimates. 
However, reinterview data are typically not available in 
panel surveys and, consequently, analysts may only be able 
to apply criteria (1), (2), and (5) above to check model 
validity. The Markov assumption is key to the MLCA 
approach. Some panel data may seriously violate this 
assumption. Fortunately, failure of Markov assumption 
appears not to be an important factor in the validity of 
MLCA estimates of CPS labor force classification error (cf. 
Table 4). 
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Estimation and Replicate Variance Estimation 
of Median Sales Prices of Sold Houses 


KATHERINE J. THOMPSON and RICHARD S. SIGMAN! 


ABSTRACT 


The U.S. Census Bureau publishes estimates of medians for several characteristics of new houses, with a key estimate being 
sales price of sold houses. These estimates are calculated from data acquired from interviews of home builders by the 
Survey of Construction (SOC). The SOC is a multi-stage probability survey whose sample design is well suited to the 
modified half-sample replication (MHS) method of variance estimation. The literature supports applying the MHS method 
to replicate sample medians to estimate the sampling variance of a median. There are several computational advantages, 
however, to using grouped data to estimate medians, with linear interpolation being used within the grouped-data interval 
containing the median. Using survey data and simulated finite populations, we compared the effects of no grouping (i.e., 
the sample median), grouping with fixed-size intervals, and grouping with data-dependent-sized intervals on medians and 
associated MHS variance estimates. We examined the mean squared errors and mean absolute errors of the median 
estimates and the relative bias and stability of the variance estimates and the coverage of the associated confidence intervals. 
We found that the data-dependent-sized intervals yielded variance estimates with the smallest bias, the best stability, and 


the best confidence intervals. 


KEY WORDS: Median; Modified half-sample replication; Survey of Construction. 


1. INTRODUCTION 


The U.S. Census Bureau publishes estimates of medians 
for several characteristics of new houses, with a key 
estimate being sales price of sold houses. These estimates 
are calculated from data acquired from interviews of home 
builders by the Survey of Construction (SOC). The SOC is 
a multi-stage probability survey whose sample design is 
well suited to the modified half-sample (MHS) replication 
method (balanced repeated replication with replicate 
weights of 1.5 and 0.5) for reasons outlined in section 3.B. 
In the near future, the SOC will move its current estimation 
and variance estimation systems to the Census Bureau’s 
re-engineered post-data-collection system, the Standardized 
Economic Processing System (StEPS). When this occurs, 
SOC will change from its current non-replicate variance 
estimation procedure to the MHS replication variance esti- 
mation procedure (Thompson 1998). Because the SOC 
variance estimation methodology is changing, we decided 
to revisit the median-estimation methodology for contin- 
uous data. Our goal was to find a median-estimation 
method with good estimation and variance estimation 
properties, given the MHS replication. 

We considered two methods of median-estimation. The 
first method uses the sample weights to estimate medians 
via empirical cumulative-distribution functions. The second 
method uses linear interpolation of grouped continuous data 
to approximate the median. The latter method is imple- 
mented in VPLX (Variances from ComPLeX Survey, Fay 
1995), the replicate variance estimation software package 
developed at the Census Bureau. 

Direct calculation of sample medians can be computa- 
tionally intensive because it requires separate sorts for each 


value of a given classification variable. An alternative esti- 
mation method is to group the continuous data into discrete 
intervals (called bins) and use linear interpolation over the 
interval containing the median. Provided that the data are 
approximately uniformly distributed over the interval 
containing the median, interpolation yields a good approxi- 
mation while being considerably less computer resource- 
demanding. However, optimal bin widths and locations may 
differ by domain and may change over time as the sample 
distributions change. 

In this paper, we compare six methods of median- 
estimation, given MHS replication: the sample median and 
five variations using linear interpolation. Section 2 provides 
a brief overview of the SOC design. Section 3 presents 
general methodology. Section 4 describes the empirical 
results from four months of SOC data that motivated the 
simulation study presented in section 5. Section 6 provides 
our conclusions and recommendations. 


2. SOC SAMPLE DESIGN 


The SOC universe contains two sub-populations: local 
areas that require building permits and local areas that do 
not. The SOC sample-units selected from the first sub- 
population comprise the Survey of the Use of Permits 
(SUP), and those selected from the second sub-population, 
the Nonpermit Survey (NP). The SUP sample comprises the 
majority of the SOC estimate. The two samples are 
multi-stage probability samples stratified by variables with 
high expected correlation with the survey’s key statistics: 
housing starts, completions, and sales. 
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The first stage of the SUP and NP sample selection is a 
subsample of 1980 design Current Population Survey 
(CPS) Primary Sampling Units (PSUs), which are conti- 
guous areas of land with well-defined boundaries. Thus, 
both surveys are conducted in the same PSUs but are other- 
wise independent samples. The PSUs were stratified within 
region by weighted 1980 population 16 years and older, 
weighted 1982 residential permit activity, and percent of 
housing in nonpermit areas. When possible, strata consisted 
of PSUs from the same state with the same metropolitan 
status. One PSU per stratum was selected. Self-representing 
(SR) PSUs were included in the sample with certainty (the 
stratum consists of one PSU). Nonself-representing (NSR) 
PSUs were selected with probability proportional to size 
(PPS) from strata containing more than one PSU. 

The second stage of SUP sample selection is a stratified 
systematic sample of permit-issuing places within sample 
PSUs (selected once a decade). These places were stratified 
by a weighted average of the ratio of permit-issuing activity 
in year i to the total US permit activity in year i (i = 78, 81, 
82). In many cases, only one second stage unit was selected. 
The third stage of SUP sample selection is performed 
monthly: each month, Field Representatives (FRs) select a 
systematic sample of building permits from the permit of- 
fices in each sampled permit-issuing place. One-to-four-unit 
building permits are selected systematically in such a way 
that an overall one-in-forty sample is achieved; five-or- 
more-unit building permits are included with certainty. The 
third-stage samples are independent by month; the first and 
second stages are not. 

The second stage of NP sample selection is a stratified 
systematic sample of small land areas (1980 Census 
Enumeration Districts, or EDs), stratified by 1980 Census 
population size. For the third stage of NP sample selection, 
field representatives completely canvass all of the roads in 
the sampled EDs (called segments). To reduce canvassing, 
a few of the larger EDs were subsegmented and one subseg- 
ment selected, or large EDs were 1-in-2 subsampled. 
Currently, there are a total of seventy-one active nonpermit 
segments. All new housing units are included in the NP 
sample with certainty. 

Median estimates are derived from the pooled SUP and 
NP samples and are calculated using a post-stratified weight 
for the SUP portion and an unbiased weight for the NP 
portion. 


3. METHODOLOGY 


A. Median-Estimation Procedures 


1. Sample Median 


One procedure for estimating the median of a popula- 
tion is to calculate the sample median from ungrouped 
data, using the sample weight to locate the median. 
This approach is recommended in Kovar, Rao and Wu 


(1988) and Rao and Shao (1996). The procedure uses 
the following steps: 


— sort the sample observations in ascending order; 


— accumulate the sum of the associated survey 
weights; 


— select the first observation for which the associated 
sum of the weights exceeds fifty percent of the total 
weight. 


2. Linear Interpolation 


Another approach for estimating the median of a 
population is to group the sample data and interpolate 
for the sample median. Woodruff (1952) provides the 
following formula for linear interpolation of a sample 
median: 


J iaeg 


M= FCN) = I+ eu G2) 


= the cumulative frequency of the characteristic 
using sample weights 


ll = lower limit of the bin containing the median 


N = estimated total number of elements in the 
population 


cf = cumulative frequency in all intervals preceding 
the bin containing the median 


f, = median class frequency (estimated total number 
of elements in the population of the interval 
containing the median) 


i= width of the bin containing the median 


This is the method used by the current SOC production 
variance estimation system for monthly estimates and is 
also the linear interpolation method employed by VPLX. 

We considered two options for setting the class size (bin 
widths) for the interpolation. The first option develops bins 
based on the specific characteristic under consideration 
using the original data. The second option linearly trans- 
forms the data to a standard scale and then uses a standard 
set of bins for every characteristic. We used the following 
linear transformation: 


_ y , 1,000 
Ul Ul Q, 
where Q, is the third quartile of the sample distribution 
(estimated using the ordered observations and sample 
weight as outlined in section 2.A.1). The interpolated 
median of the X’ is multiplied by (Q,/1,000) to obtain an 
estimated median on the original scale [If the distribution 


contains negative values (e.g., a distribution of net income), 
then use X;" = (X;, ~X)) * 1,000/0,(X, ~X)), where Xi) is 


(2) 
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the first order statistic and Q,(X, —X,,)) is calculated from 
the distribution of (X;, ~X i). To obtain an estimated 
median on the original scale, multiply the interpolated 
median by (Q,(%; ~Xiqy)/ 1,000) and add Xay] This 
procedure is equivalent to simply dividing the original 
sample from 0 to Q, into x bins of equal width and placing 
the remainder of the data into one bin which, by design, is 
much larger than the others. 

This procedure is designed for symmetric or positively 
skewed distributions (usually the case with economic data). 
The data in the last bin is not used to estimate the median 
because it is greater than Q,, which is expected to be far 
from the median. If we based the linear transformation on Q, 
(the first quartile), the bin containing the median might be 
very close to the lowest bin in the distribution. In this case, 
the difference in variability between an interpolated median 
and the sample median would be small. 

Using the original data to develop medians has the 
advantage of producing production-ready estimates and 
SEs. Determining the appropriate fixed bin width is 
difficult, however. As the bin widths get small (approach 
width 1), the variance estimates become more unstable. As 
the bin widths increase, the bias of the estimate due to 
interpolation increases. The “optimal” bin size balances 
variance estimate stability and bias. Unfortunately, the 
optimal bin width may not remain constant between 
samples. Often, the distributions change over time, and the 
bins widths/locations in the sample should reflect this 
change in scale. Moreover, the optimal bin width may be 
different for different values of a classification variable: for 
example, the optimal bin width for the Midwest’s sales 
price is probably different from the optimal bin width for 
the South’s sales price. 

The desire to have the width of the bin depend on the 
sample motivated the linear transformation. The “standard” 
bin widths used for the transformed data less than Q, are 
not standard on the untransformed scale: the bin width is 
data-dependent. Using the linearly transformed data re- 
quires more bookkeeping in terms of scaling constants but 
easily allows for changes in the scale and shape of the 
distribution. 

Figures 1 through 4 illustrate the effect of the linear 
transformation on the bin widths and location for two distri- 
butions. Figures 1 and 2 present a distribution that has a 
large spread of data values, including a few very large 
observations. Figures 3 and 4 present a distribution con- 
sisting of primarily small data values. 

Figure 1 presents a histogram of the original distribution 
for houses sold with conventional financing, with bin width 
of $25,000 [Note: the bin size was selected purely for 
presentation convenience, since this is a long-tailed 
distribution]. The median of this distribution is $167,130, 
and Q, is $225,000. Figure 2 presents the histogram of the 
linearly transformed distribution with bin width of 50. In 
this example, the transformed bins of width 50 are 
equivalent to bins of width $11,250 on the original scale 
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(($225,000/1,000)*50). Recall that the original-data bin 
sizes considered are $1,000 and $2,000. Thus, the 
transformed-data bins of width 4 would have a width of 
$900 on the original untransformed scale. Notice the large 
“spike” at the last bin, which contains all of the sample 
greater than Q,. 

These figures also illustrate the differences in distribu- 
tion of sample sizes across bins between the two methods. 
Using fixed bin widths with the original data results in quite 
variable bin sample sizes (see Figure 1). In contrast, by 
design the sample sizes within the data-dependent bins are 
much more uniform for all but the last bin (see Figure 2). 
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Original Distribution of Sales Price of Houses Sold 
With Conventional Financing Bin Width = $25,000 


Figure 1: 


Figure 2: Transformed Distribution of Sales Price of Houses Sold 
With Conventional Financing Using Bin Width = 50 Bin 
Width on Untransformed Scale = $11,250 


Figure 3 presents a histogram of the original distribution 
of houses sold with FHA loans, with bin width of $4,000 
(again, the bin width is chosen for presentation conve- 
nience). The median of this distribution is $108,280, and 
Q, is $124,990. Figure 4 presents the histogram of the 
linearly transformed distribution with bin width of 50. In 
this example, the transformed bins of width 50 are 
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equivalent to bins of width $6,250 on the original scale, and 
the transformed-data bins of width 4 would have approx- 
imate width $500 on the original untransformed scale. 
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Figure 3: Original Distribution of Sales Price of Houses Sold 
With FHA Loans Bin Width = $4,000 


Figure 4: Transformed Distribution of Sales Price of Houses 
Sold With FHA Loans Using Bin Width = 50 
Bin Width on Untransformed Scale = $6,250 


Figures 1 through 4 demonstrate the flexibility of the 
bins developed for linearly-transformed data. The bin size 
on the untransformed scale expands or contracts, depending 
on the spread of the data. Moreover, the data-dependent bin 
sample sizes are less variable compared to those associated 
with fixed bins. 

To evaluate the first interpolation option (original-data- 
interpolated medians), we used two different sets of bin 
widths (classification sizes): bins of size $2,000 (the same 
bin width used in the current production variance estimation 
system) and bins of size $1,000. [Note: The VPLX variance 
estimation software would not allow any bin size smaller 
than 1,000 because the number of classes exceeded the 
allowable array range.] After examining several months of 
sales price estimates for the total U.S., we assumed that 
median sales price would always be larger than $36,000 and 
smaller than $550,000, so the first original-data classifi- 
cation is always (low — 35,999) and the last original-data 


classification is always (550,000 — high): this yields 257 
bins of size $2,000 or 514 bins of size $1,000, plus one bin 
of size $36,000 and one bin whose width depends on the 
largest observation in the sample. One obvious problem 
with the locations of these bins is the potential effect of 
inflation. It is conceivable that within special financing 
categories or certain regions, the median sales price for 
houses sold could approach $550,000, and the interpolation 
would fail as a consequence. 

To evaluate the second interpolation option 
(transformed-data-interpolated-medians), we used three 
different sets of bin widths: bins of size 4, 25, and 50. The 
bins of size 4 were chosen to be analogous to the bins of 
size 2,000 in terms of the number of bins. There are 250 
bins of size 4 for the transformed data less than Q,, and one 
larger bin containing all data greater than Q,. The selection 
of widths 25 and 50 was somewhat arbitrary: we chose bin 
size 50 to get a total of twenty bins for the data less than Q, ; 
and we chose bin size 25 to examine the effect of doubling 
the number of bins/halving the width of the bins for data 
less than Q,. The transformed-data median is always less 
than 1,000, so the last transformed-data classification is 
always (1,000 — high). Thus, by definition the last bin 
contains up to twenty-five percent of the data and is 
considerably wider than the other bins. 


B. Variance Estimation 


We used the Modified Half-Sample (MHS) replication 
method (Fay 1989 and Judkins 1990) to estimate the 
variance of a median as supported in the literature (e.g., 
Rao, Wu, and Yue’ (1992); Rao and Shao (1996); 
Kovacevic and Yung (1997) for balanced repeated repli- 
cation; and Judkins (1990) for MHS replication). MHS 
replication is a variation of the “traditional” balanced half- 
sample variance estimation described in Wolter (1985, 
110-152). Balanced half-sample replication (BRR) is a 
variance estimation method designed for a two-PSU per 
stratum design. With BRR, a half-sample replicate is 
formed by selecting one unit from each pair and weighting 
the selected unit by 2 (so that it represents both units). 
Thus, estimates for every PSU are included in each 
replicate although half are weighted by zero. Replicates 
(half-samples) are specified using a Hadamard matrix. See 
Wolter (1985, 114-115) for a detailed description of the 
replicate formation procedure using Hadamard matrices. 
MHS replication uses replicate weights of 1.5 and 0.5 in 
place of the 2 and 0. The standard error for a median 
estimate using MHS replication is given by 


SE(Med) = 


R 
> (Med, - Med,)” 
r=1 


where the r subscript refers to the replicate r median 
estimate (r=1,2, ..., R) and the O subscript refers to the full 
sample the median estimate. This expression contains a four 


(4) in the numerator because the MSE of the replicate 
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estimates is too small by a factor of 1/(1-0.5)*. See Judkins 
(1990). 

Neither the SUP nor the NP designs are two-sample- 
unit-per-stratum designs. At the first stage, one PSU per 
stratum is selected. The second and third stages are sys- 
tematic samples, and often only one unit per stratum was 
selected at the second stage. A common approach used to 
address the one sample-unit per stratum problem is to 


— “split” the SR sample-units into two panels per sample- 
unit using the original sampling methodology; 


— form collapsed strata by pairing two (or three) “similar” 
NSR sample-units; and 


— apply the half-sample approach in such a way that the 
elements contributing to the half samples are panels 
within sample-units for SR sample-units and are the 
first stage sample-units (PSUs) within collapsed strata 
for NSR sample-units. 


The current SOC production variance system uses a 
Keyfitz estimator (a paired difference estimator) for NSR 
sample and an approximate sampling-formula estimator for 
SR sample to produce level estimate variances (Luery 
1990). Because SOC methodologists had already collapsed 
NSR strata for their paired difference estimator, a BRR-like 
application was a logical extension of the pre-existing vari- 
ance estimation structure. For MHS replication, we sort per- 
mits within predetermined sample-unit groups in SR units 
by geography and authorization date and systematically 
split the ordered sample into two panels as suggested in 
Wolter (1985, 131). Although this is essentially the only 
approach available for the SOC design, this method may 
not provide the correct variance estimates since units in 
both panels are correlated (in the original half-sample meth- 
od, the two PSUs in the stratum are assumed independent). 
For more details on the replicate assignments, see 
Thompson (1998). 

The SOC production system uses the Woodruff method 
(Woodruff 1952) to estimate the standard error of a median. 
The Woodruff method uses the estimated SE of a pro- 
portion p (p = 0.50 for median-estimation) and projects the 
interval (p + SE(p)) through the cumulative frequency 
distribution to obtain the lower limit of a 62.86 percent con- 
fidence interval for the median (the SE(p) can be estimated 
using replicate methods). The SE of the median is then 
estimated by subtraction. This methodology has had mixed 
success in the past according to SOC survey analysts. 


4. EMPIRICAL DATA RESULTS 


Initially, we used four months of SOC sample data to 
examine the variances of the median-estimation methods 
for sales price of sold houses: March 1997, May 1997, June 
1997, and July 1997. We produced medians by region and 
by type of financing. We used the same weight used by the 
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SOC production estimation and variance systems (post- 
stratified for SUP sample and unbiased for NP sample), 
pooling both surveys’ data to obtain medians. Each set of 
variance estimates was produced using 200 replicates. 

We found that the six median-estimation methods 
produced very similar estimates, but yielded three distinct 
sets of SEs: one set for the sample median, one set for the 
original-data-interpolated medians (fixed bin width), and 
one set for the transformed-data-interpolated medians (data- 
dependent bin width). There was no clear relationship be- 
tween bin width and SE estimates within the two sets of 
interpolated medians. Indeed, within type of data (original 
or transformed), the SEs were all very close. Clearly, there 
was a linear transformation and an interpolation effect. 
None of the median-estimation methods yielded standard 
errors resembling the published standard errors, so there 
was no available argument for publication consistency. 

Moreover, there is some evidence that the Woodruff 
method publication SEs are underestimates or are at least 
inappropriate for the sample design used. Kovar, Rao, and 
Wu (1988) compared Woodruff SEs and BRR standard 
errors and found that the two methods had similar 
properties except for the case of stratified samples, where 
the strata are based on highly correlated separate variables 
(such as the SOC design). In this case, the Woodruff SE is 
often too small, and they concluded that “the BRR... 
methods (sic) are more robust to different population 
structures, since the error is extracted directly from the 
replicates.” When the production system Woodruff SEs 
used the directly-calculated SE(p), the Woodruff SEs were 
generally smaller than the replicate SEs. 

The empirical results left us in a quandary. We had three 
distinct sets of variance estimates, and no “gold standard” 
against which to measure them. Because our empirical 
results were inconclusive, we conducted a Monte Carlo 
simulation study to evaluate the properties of the MHS 
variance estimates produced from the different median 
estimators. 


5. SIMULATION STUDY COMPARISON 


A. Procedure for Simulation Study 


We created four finite artificial populations based on a 
data analysis of four SOC sample populations: one type-of- 
financing population (Conventional Financing) and three 
regional populations (Midwest (Region 2), South (Region 
3), and West (Region 4)). These populations represented a 
variety of the types of SOC populations from which 
estimates are produced. Note that the SOC type-of- 
financing population is not independent of the SOC-region 
populations. 

To approximate the finite population of sales price for 
houses sold, we generated w, records for each sample-unit 
i, where w, is the sample weight associated with unit 1. The 
distributions of sales price for single-unit sold houses could 
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be approximated by lognormal distributions. The lognormal 
distribution has the probability density function 


ae 1 _ Gog(y-8)-9)\2 
(OR xp Gee) 
y-9 216 2 
for 8<y<o 


where 6 is the threshold parameter, ¢ is the scale parameter, 
and o is the shape parameter. 

From our models, we generated four simulated finite 
bivariate populations with expected correlation p = 0.6 
using the method outlined in Naylor, Balintfy, Burdick and 
Chu (1968, 99). The first of the two variables in each 
population represented sales price of sold houses and was 
obtained by generating a random normal variable with mean 
¢ and variance o using the parameters determined above, 
then exponentiating and shifting by the appropriate location 
parameters (8). The second variable was used to form strata 
and first stage clusters. This variable had a marginal 
standard normal distribution and was obtained by indepen- 
dently generating a second standard random normal value, 
multiplying it by 0.8, and adding this term to 0.6 x the 
standard normal random variable used to generate the sales 
price variable. Percentiles, sample skewness, and sample 
kurtosis of each simulated population’s sales price variable 
were very close to the corresponding statistics in the 
original population, especially when outliers were deleted 
using the resistant outer fences rule described in Hoaglin 
and Iglewicz (1987). Each population’s size was the N 
estimated from the sample populations. Model parameters, 
sample correlations (between simulated sales price and 
stratifying variable), population size (4), and sample sizes 
(n) are reported in Table 1. 

After generating the finite populations, we formed 50 
equal sized strata in each population, then selected two sets 
of samples for two different survey designs: 


— The first design is patterned after the SUP sample of 
permits for four-or-less-housing units in SR permit 
offices in SR PSUs (approximately 28% of the SOC 
sample). In this study, we selected 5,000 stratified 
without-replacement random samples from each simu- 
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— The second design is patterned after the SUP sample of 
permits for four-or-less-housing units in NSR permit 
offices in SR PSUs and in SR permit offices in NSR 
PSUs (approximately 40% of the SOC sample). In this 
study, we selected 5,000 two-stage samples from each 
simulated population. The first stage is stratified 
without-replacement random sample of two PSUs per 
stratum (N, =5). The second stage is a systematic 
sample of units within PSUs. Because all PSUs are the 
same size, this study does not take the SOC PPS sam- 
pling into account and does not include the collapsing 
of first-stage units. The MHS replication uses the first- 
stage sample units (PSUs) within the same strata. The 
replicate weights do not account for large sampling 
fractions at the first stage of selection as recommended 
in Wolter (1985, 122), so all of the variance estimates 
are probably upwardly biased. 


We did not attempt to simulate the SUP sample of 
permits for four-or-less-housing units in NSR PSUs and 
NSR permit offices (a three-stage sample, approximately 
25% of the SOC sample); the SUP sample of permits for 
five-or-more housing units (approximately 2% of the SOC 
sample); or the NP sample of EDs (approximately 5% of 
the SOC sample). The three-stage sample, although non- 
negligible in SOC, is rarely used by other surveys at the 
Census Bureau, and the other two sectors of the SOC 
design do not contribute enough to the estimates to warrant 
a separate investigation. 

To examine the precision of each median-estimation 
procedure over repeated samples, we estimated empirical 
Mean Squared Errors (MSE) and Mean Absolute Errors 
(MAE) from the 5,000 samples for: 


SM: the sample median of each half-sample 


102000: interpolated medians using original data, bins of 
size 2,000 (fixed bin width) 

101000: interpolated medians using original data, bins of 
size 1,000 (fixed bin width) 


IT4: interpolated medians using linearly transformed 
data, bins of size 4 (data dependent bin width) 


; : ' IT25: interpolated medians using linearly transformed 
lated population using the same sampling rate in each data, bins of size 25 (data dependent bin width) 
stratum. To perform MHS replication, we sorted the ? P 
sample within each stratum by stratifying variable and IT50: _ interpolated medians using linearly transformed 
then systematically split the sample into two panels. data, bins of size 50 (data dependent bin width) 

Table 1 
Characteristics of Simulated Populations and Sample Sizes of Stratified Samples 
pales Econ enieiets Mtecifer Salsa Pa Sawa 
Population Distribution ) o C p N n 
Conventional Financing lognormal 27,578 0.4895 11.84 0.57030 25,150 500 
Midwest lognormal 31,801 0.5957 11.69 0.55835 6,500 150 
South lognormal 29,414 0.5549 TESS 0.55929 14,550 300 
West lognormal 53,781 0.5822 Pleo 0.55525 11,550 250 
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Table 2 
Median, Third Quartile, and Bin Widths on Original Scale for Transformed Simulated Data 
Population Median Biigy ict 
4 25 50 
Conventional Financing 167,173 222,263 889 S57/ eS 
Midwest (Region 2) 1515312 210,647 843 5,266 10,532 
South (Region 3) 133,745 180,868 723 A\ Syo) 9,043 
West (Region 4) 162,130 214,320 857 5,358 10,716 


The linear transformation was performed once for 
procedures IT4, IT25, and ITSO. The original data were 
transformed using the full sample Q,, and these trans- 
formed data were assigned to the half-samples (including 
replicate 0, the full sample). Table 2 provides the median 
and third quartile of each finite population, along with the 
bin widths on the original scale for the transformed data. 

We calculated M(G), the empirical MSE of median- 
estimation procedure i as 


= 6%(C,) + bias “(G) (5.1) 
where G, is the estimated median for sample r and estimator 
i, ¢, is the average of the ¢,, and ¢, is the population 
median. This is the empirical MSE described in Judkins 
(1990). 

We calculated the Mean Absolute Error (MAE) of each 
median-estimation procedure i as 


Dacre sn 
MAE(6,) = — eer (5.2) 


as defined in DeGroot (1986, 209-211). 

To compare the variance estimation properties of the 
different median-estimation methods, we calculated an 
MHS variance estimate (v,,) corresponding to each median- 
estimation procedure i from 1,000 of the 5,000 samples. 
These variance estimates were compared in terms of 


Relative bias ()_ v,/1,000)/M(G) - 1 
ja 


Relative stability o> (, -M(G))/ 1,000)]'/M(C,) 

j=l 
Error Rate Number of samples where (G< 0,; or G > 
8y;)/1,000 where 


0, is the lower end of a 90% confidence interval, and 
Oy; is the upper end of a 90% confidence interval 


These criteria are used in Kovar, Rao, and Wu (1988) 
and in Rao and Shao (1996). The relative bias is a measure 
of the bias of the variance estimate as a proportion of the 
true MSE. The stability is a measure of the variance of the 
variance estimates; it approximates a c.v. of the variance 
estimate v;. Note that the relative stability is not the relative 
MSE defined in Wolter (1985, 297) which uses the 
squared-MSE in the denominator. With an “optimal” vari- 
ance estimator, both the relative bias and relative stability 
will be near zero, and the error rate will be ten percent. 


B. Results 


1. Comparison of Median-estimation Procedures 


Table 3 presents the empirical root MSE, standard 
error, the bias, and the MAE for each median-esti- 
mation procedure from both simulation studies. Each of 
these statistics was calculated from 5,000 samples. 


These results reinforced our suspicions from the 
empirical data analysis described earlier. At least for 
sales price, all six median-estimation procedures per- 
form approximately equally well, with approximately 
equal root-MSEs and MAEs between procedures in 
each population. 


2. Comparison of MHS Replication Variance Estimation 
Properties of Median-Estimation Procedures 


When we examined the variance estimation properties 
for each procedure, the results were quite different. As with 
our empirical data analysis, we had three very distinctive 
sets of results. Table 4 summarizes the three different 
comparison measures for the variance estimates in the four 
populations. The numerators for the relative bias and stabil- 
ity and the coverage rates are based on 1,000 samples. The 
denominator for the relative bias and stability (“truth”) are 
based on 5,000 samples. An asterisk (*) in the last column 
of Table 4 indicates that the error rate is significantly differ- 
ent from the nominal error rate of 0.10 using the normal ap- 
proximation to the binomial distribution at the 90% confi- 
dence level. 
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Table 3 
Empirical Root MSE, Standard Error, Bias, and MAE for Median-Estimation Procedures 


Median- Unclustered Single-Stage Sample Clustered Two-Stage Sample 

See ae mage ea ee SE Bias MAE as SE Bias MAE 
Conventional SM 3,345 3,345 -12 2,671 3,389 3,374 324 23S 
Financing 102000 3,320 3,316 161 2,698 3,346 3,341 189 2,685 
101000 3,387 3,368 -354 2,642 3,43] 3,420 -278 2,774 

IT4 3,351 3,340 273 2,673 3,378 3,364 Bit INS) 

IT25 3,304 3,293 276 2,617 3,331 3y3)1 322 2,664 

ITSO 3,282 3,265 329 2,606 3,305 3,283 375 2,636 

Region 2 SM 6,316 6,287 -598 4,966 6,273 6,228 -753 4,959 
Midwest 102000 6,276 6,275 -127 4,992 6,335 6,207 -1,271 5,029 
101000 6,343 6,297 -767 4,939 6,526 6,280 -1,774 5,204 

IT4 6,372 6,363 328 5,004 6,294 6,228 -908 4,979 

IT25 6,273 6,272 127 4,937 6,270 6,154 -1,199 4,971 

ITSO 6,220 6,218 160 4,936 6,224 6,114 -1,164 4,966 

Region 3 SM 3,670 3,658 301 2931 3,835 Shee 796 3,054 
South 102000 3,708 3,669 539 2,998 3,796 3,739 656 3,011 
101000 3,742 3,740 101 2,941 3,809 3,804 212 3,066 

IT4 3,718 3,662 639 2954 3,814 3,736 766 3,028 

IT25 3,699 3,638 669 2,924 395 3,711 787 292 

ITSO 3,692 3,616 745 2,912 3,778 3,680 856 2,970 

Region 4 SM 4,385 4,382 -140 3,509 4,394 4,351 616 3,506 
West 102000 4,425 4,421 185 3,578 4,362 4,339 449 3,487 
101000 4,477 4,469 -258 3,530 4,411 4,410 -57 31535 

IT4 4,414 4,403 318 3,514 4,383 4,342 599 3,494 

IT25 4,376 4,364 315 3,460 4,334 4,296 573 3,439 

ITSO 4,367 4,350 391 3,455 4,320 4,271 644 3,436 


In both studies, the variance estimates of the trans- 
formed-data-interpolated medians perform best in terms of 
relative bias and stability. Specifically, 


— The variance estimates of the transformed-data- 
interpolated medians (IT4, IT25, IT50) have the 
smallest relative bias. The difference in estimation 
method is quite pronounced in three of the four popula- 
tions, where the largest relative bias of the transformed- 
data-interpolated medians is less than one-half the size 
of the smallest relative bias of the original-data- 
interpolated and sample medians. These results are sur- 
prisingly strong for the two-stage clustered design, 
since the variance estimates are expected to be biased 
upwards (see section 5.A); 


— The variance estimates of the interpolated medians had 
the best stability. The variance estimates of the sample 
median had the poorest stability in all four populations. 
This result was expected due to the smoothing effect of 
interpolation. Again, the transformed-data-interpolated 
medians generally performed better than the original- 
data-interpolated medians, although the difference is 
not as pronounced as in the case of relative bias. Gen- 
erally, the stability is close with all three bin widths for 
the transformed-data-interpolated medians. 


The results for each median-estimation procedure’s 
confidence interval coverage are not as consistent, varying 
by design. With the single-stage unclustered design, the 


confidence intervals constructed from transformed-data- 
interpolated medians and SEs have the best coverage. In 
each population, the data-dependent bins (all widths) yield 
close to nominal or better coverage; in fact, none of these 
error rates is statistically different from the nominal 10%. 
The confidence intervals constructed from original-data- 
interpolated medians and SEs are extremely conservative. 
Here, the positive bias in the variance estimates makes these 
intervals unnecessarily wide, thereby reducing the power to 
make interesting findings. The coverage with the sample 
median is erratic. 

Some of these coverage patterns are repeated in the two- 
stage clustered design. Again, the coverage with the sample 
median is erratic, and the coverage rates for the confidence 
intervals constructed from original-data-interpolated medi- 
ans are better than nominal (although only significantly 
better than nominal in two populations). The error rate 
pattern is quite different for the transformed-data-inter- 
polated medians. In all but the Region 4 population, the 
coverages rates for the three procedures are worse than 
nominal. However, with bins of widths 4 and 25, only one 
error rate is significantly larger than 10%; for bins of width 
50, two of these three error rates are significantly larger 
than 10%. All of the interpolated-data-medians have sig- 
nificantly smaller than nominal error rates in the Region 4 
population; consistent with the other population’s results, 
the error rates for the original-data-interpolated medians are 
the farthest from 10%. 
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Table 4 
Relative Bias and Relative Stability for Variance Estimates, and Error Rates for 90% Confidence Intervals 


Median- 
Population Estimation . 
: Procedure Relative Bias SPR 

Conventional SM 0.19 0.69 
Financing 102000 0.25 0.35 
101000 0.21 0.32 

1IT4 0.06 0.25 

IT25 0.07 0.25 

ITSO 0.05 0.26 

Region 2 SM 0.57 1.24 
Midwest 102000 0.33 0.44 
101000 0.30 0.42 

IT4 0.15 0.41 

1T25 0.16 0.40 

ITSO 0.15 0.42 

Region 3 SM 0.30 0.88 
South 102000 0.31 0.42 
101000 0.29 0.40 

1T4 0.04 0.29 

IT25 0.02 0.28 

ITSO 0.01 0.29 

Region 4 SM 0.39 0.98 
West 102000 0.32 0.42 
101000 0.29 0.39 

IT4 0.11 0.32 

IT25 0.10 0.31 

ITSO 0.08 0.31 


In both studies, the transformed-data-interpolated 
medians have the best variance estimation properties in 
terms of relative bias and relative stability by a large 
margin, regardless of bin width. And, in both studies, the 
transformed-data-interpolated medians using bins of width 
4 or width 25 have excellent confidence interval coverage. 
Since the transformed-data-interpolated-medians using bins 
of width 50 or width 25 yielded the “best” estimators in 
terms of root-MSE and MAE in both studies, using linear 
interpolation on transformed data with bins of width 25 
appears to be the best median-estimation procedure in terms 
of estimation and variance estimation properties. 


6. CONCLUSION 


We explored the effect of using variations of two 
different methods of estimating the median sales price of 
sold houses: direct estimation versus linear interpolation. 
Linear interpolation requires classifying continuous data 
into bins of standard width. This width can be arbitrary, can 
differ greatly by domain, and may change as the sample 
distribution changes over time. The linear transformation 


Unclustered Single Stage Design 


Clustered Two-Stage Design 


Error Rate Relative Bias SOEs ay . Error Rate 
Stability 
11.0% 0.11 0.58 1Sal So 
6.9%* 0.25 0.37 9.0% 
7.0%* 0.19 0.33 9.3% 
10.0% 0.06 0.27 11.3% 
10.9% 0.06 0.27 11.8%* 
9.5% 0.05 0.28 12.1%* 
7.3%* 0.41 1.07 7.9%* 
6.9%* 0.23 0.35 8.6% 
7.0%* 0.17 0.30 8.7% 
10.1% 0.14 0.41 11.5%* 
9.8% 0.11 0.37 10.4% 
9.0% 0.11 0.40 10.4% 
12.4%* 0.15 0.71 11.1% 
6.7%* 0.28 0.39 Teo vor 
6.7%* 0.27 0.38 7.3%* 
11.0% 0.01 0.28 10.8% 
11.0% -0.01 0.27 11.3% 
11.1% -0.02 0.28 11.9%* 
8.9% 0.25 0.79 8.6% 
6.2%* 0.31 0.41 5.2%* 
6.2%* 0.28 0.38 57270" 
8.6% 0.10 0.31 7.6%* 
9.4% 0.09 0.30 ES To 
9.5% 0.08 0.31 


8.3%* 


based on the third quartile appeared to correct this problem. 
With the transformed data, the bins’ widths and locations in 
the distribution change depending on the data. 

Our empirical results indicated that the choice of method 
has a pronounced impact on the variance estimates given 
MHS replication. Our simulation study examined the 
properties of the different median-estimation procedures on 
the MHS replicate variance estimates. In all four simulated 
populations, the transformed-data-interpolated medians 
(data dependent bin widths) performed the best, usually by 
a wide margin. Most critically, this method greatly reduces 
the overestimation of the variance. Using bins of width 25 
on the transformed scale (41 bins total) yielded the best 
median sales price estimates and variance estimates, given 
MHS replication and is our recommended method for the 
Survey of Construction. 

The recommended method has several advantages. First, 
it is adaptive. It works well for a variety of distributions, 
because the bin widths themselves depend on the distribu- 
tion at hand. Second, it saves computing resources by 
avoiding sorting half-samples. Third, the data-dependent- 
intervals can be easily incorporated into generalized survey- 
processing software. Finally, it gives better estimates and 
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MHS replicate variance estimates (at least for sales price of 
sold houses). We expect that these results are generalizable 
for other continuous distributions as well, although obvi- 
ously this conjecture should be tested on other data sets. 
Other areas for future research include examining the 
relationship between sample size and precision of the 
median estimates, examining alternative bin sizes, and 
exploring the robustness of the recommended procedure 
with different replicate variance estimation procedures. 
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The Impact of Different Rotation Patterns on the Sampling 
Variance of Seasonally Adjusted and Trend Estimates 


C.H. McLAREN and D.G. STEEL! 


ABSTRACT 


Many economic and social time series are based on sample surveys which have complex sample designs. The sample design 
affects the properties of the time series. In particular, the overlap of the sample from period to period affects the variability 
of the time series of survey estimates, and the seasonally adjusted and trend estimates produced from them. The Census 
X11 and X11ARIMA packages are commonly used to produce seasonally adjusted estimates and can also be used to 
produce estimates of trend. This paper considers the implications of different overlap patterns on the sampling variance 
of seasonally adjusted and trend estimates obtained from time series based on sample surveys. 


KEY WORDS: X11; X11ARIMA; Seasonal adjustment; Trend estimation; Rotation patterns. 


1. INTRODUCTION 


Many important time series are based on repeated sample 
surveys which have complex patterns of sample overlap from 
period to period. The use of sampling means that the 
estimated time series have a component of variability due to 
sampling errors and for many series this will be a major 
source of variability. The sample design, in particular the 
overlap pattern, affects the variability of the time series of 
survey estimates. 

Increasingly, analysis of time series is concentrating on 
assessing underlying patterns of change or trends based on 
analysis of the seasonally adjusted series. Most government 
statistical agencies have calculated seasonally adjusted series 
for many years. Kenny and Durbin (1982) noted that policy 
analysts frequently say that they are more interested in 
underlying trends than following irregular fluctuations in the 
de-seasonalized monthly values. A similar view is expressed 
by Smith (1997). For more than 10 years the Australian 
Bureau of Statistics (ABS) has published series of trend 
estimates obtained by applying Henderson Moving Averages 
(HMAs) (Henderson 1916) to the seasonally adjusted series 
to smooth out the irregular components of the series (ABS 
1987). Other government statistical agencies also produce 
trend estimates using a variety of methods (Knowles 1997). 
Since seasonally adjusted and trend estimates are obtained 
by processes applied to the original series, they are also 
influenced by sampling errors. Bell and Kramer (1999) note 
that the variance of seasonally adjusted estimates will often 
be dominated by the contribution from sampling error. Some 
series are based on independent samples over time, but 
usually the samples used have a degree of overlap from 
period to period to reduce costs and the standard errors of 
estimates of change between two consecutive time periods 
(Kish 1998). 
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A key issue in the development of the design of a re- 
peated survey is the rotation pattern, that is, the pattern of 
a selected unit’s inclusion in the survey over time, which 
will determine the sample overlap. The aim of this paper is 
to determine the effects of the rotation pattern used on the 
sampling variance of the estimated seasonally adjusted and 
trend series obtained using the Census X11 method 
developed by Shiskin, Young and Musgrave (1967) and 
X11ARIMA developed by Dagum (1980 and 1988). We 
will focus on the estimates of the level and one period 
change in the seasonally adjusted and trend estimates. 


2. ROTATION PATTERNS 


Consider a univariate time series with values y,, 
t=1,...7, obtained from a repeated sample survey. The 
observed value at time ¢ is related to the true value of the 
series in the finite population, Y,, by 


Vee 


where e, is the sampling error. The series Y, is thought to 
consist of trend-cycle, seasonal and irregular components 
LS, and J,, so that 


VT Oder 


In some cases a multiplicative decomposition may be 
more appropriate. Many statistical agencies produce sea- 
sonally adjusted series by attempting to estimate S, and 
remove it from the series, usually using some combination 
of linear filters. Most commonly used is the Census X11 
method developed by Shiskin ef al. (1967) and X11ARIMA 
developed by Dagum (1980 and 1988). Findley, Monsell, 
Otto, Bell and Pugh (1998) described further enhancements 
embodied in X12ARIMA. The ABS also publishes trend 
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estimates obtained by applying HMAs to the seasonally 
adjusted series and encourages users to base their interpre- 
tation of the series on these trend estimates (Linacre and 
Zarb 1991; ABS 1993). The HMAs were originally derived 
by Henderson (1916) for use in actuarial work and are used 
within X11, X11ARIMA and X12ARIMA to de-trend series 
for seasonal adjustment purposes. Kenny and Durbin (1982) 
and Gray and Thomson (1996) explain the derivation of the 
HMaAs. Users can also produce trend estimates by applying 
filters to the published seasonally adjusted estimates. Kenny 
and Durbin (1982) noted that there is no unique definition of 
trend and that different filters may be used according to the 
degree of smoothness and sensitivity required. Knowles and 
Kenny (1997) investigated methods of trend estimation for 
official statistical series. For monthly series they recom- 
mended the use of HMAs, with the length of the filter being 
13 or 23 depending on the volatility of the series in question. 

The autocorrelation structure of the observed series is 
determined by the autocorrelation of the series Y, and e,, 
which will then affect the estimates of the trend, seasonally 
and irregular components. The covariance structure of the 
sampling error series, e,, can be estimated from the unit 
level survey data. By obtaining such estimates, it is possible 
to obtain estimates of the sampling variance of the estimated 
trend, seasonally adjusted and irregular series. Various meth- 
ods for doing this have been proposed; for example Steel and 
DeMel (1988) considered the effect of linear filters on the 
spectrum of the sampling error series and Wolter and 
Monsour (1981) used an approach based on the effect of 
linear filters on the autocovariance function. Sutcliffe (1993) 
adopted a similar approach using a linear approximation to 
the X11 procedure. Pfeffermann (1994) proposed a method 
which develops an estimate of sampling error directly from 
the estimated time series using various simplifying assump- 
tions. These approaches do not explicitly model the time 
series. Other authors, for example Bell and Wilcox (1993), 
Tiller (1992), Burridge and Wallis (1985) and Hausman and 
Watson (1985), considered explicit ARIMA models for both 
the true series and the sampling error series, and concen- 
trated on the estimation of the parameters of the models. 
These papers do not consider the effect of different rotation 
patterns and concentrate on producing estimates of the va- 
riances of seasonally adjusted estimates for the particular 
rotation pattern used. 

The rotation pattern used in the survey will affect the au- 
tocorrelation structure of the sampling error series and hence 
the sampling variance of the original, seasonally adjusted 
and trend estimates. Several considerations are taken into 
account in deciding upon a rotation pattern. High sample 
overlap between consecutive periods reduces the sampling 
variance of estimates of change between the periods and 
high sample overlap between periods 12 months apart 
reduces the sampling variance of estimates of annual change. 
The first occasion that a selected unit is included in the 
survey is usually the most expensive. By keeping selected 
units in the survey for longer the cost of the survey is 
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reduced. This leads to rotation patterns in which a selected 
unit is included every period for as long as possible. How- 
ever, a selected unit must eventually be rotated out of the 
survey. Besides the ethical consideration of spreading re- 
spondent load, there is the possible deterioration in re- 
sponse rate and quality of data reported if the same unit is 
included for a large number of occasions (see Kalton and 
Citro 1993, for a discussion of these issues). 

Rotation patterns vary in terms of the number of times a 
unit is included in the survey and the time interval between 
inclusions. We concentrate on monthly labour force surveys 
(MLFSs). The rotation patterns used in practice are special 
cases of the a-b-a(m) rotation patterns where selected units 
are included for a consecutive months, removed from the 
survey for b months then re-included for a further a months. 
The pattern is repeated so that selected units are included 
for a total of m occasions. Rao and Graham (1964) consid- 
ered the estimation of the finite population means and totals 
for this class of rotation patterns. The United States Current 
Population Survey (CPS) uses a 4-8-4(8) pattern (Fuller, 
Adam and Yansaneh 1992). Putting b = 0 gives an in-for-m 
rotation pattern in which selected dwellings are included for 
m months after which they are removed from the sample. 
The case m=6 corresponds to the Canadian rotation 
pattern (Singh, Drew, Gambino and Mayda 1990) and 
m = 8 corresponds to the Australian pattern (ABS 1992). 
Steel (1997) noted that the British quarterly labour force 
survey approximately corresponds to a monthly survey with 
a 1-2-1(5) rotation pattern. 

We consider the sampling variance of the seasonally ad- 
justed and trend estimates associated with the rotation 
patterns currently used in MLFSs and a number of rotation 
patterns that, while not currently used, may have some 
desirable properties. This will give an indication of which 
rotation patterns are better in terms of the component of the 
variability of the estimated series that is affected by the 
sample design. 


3. SAMPLING VARIANCE OF SEASONALLY 
ADJUSTED AND TREND ESTIMATES 


Let y, be the vector containing the values of the time 
series of survey estimates up to time 7 and Y,, be the vector 
containing the true population values. The sampling vari- 
ance of the original series is denoted by V(y ,| ¥ ,). Consid- 
er a linear filter which is used to obtain values from y ,. by 
applying a vector of filter weights w,. The filter weights 
are non-random and have no connection with the survey 
weights used in calculating the survey estimates y,. The 
filter weight vectors w, depends on the time period for 
which the filtered value refers. The weights are constant 
within the body of the series but may be modified at the 
beginning and end. The filtered value at time ¢ is 


Y,=WiYr- (1) 
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Then 
Vy, | Y,)=w, Vy, | Y,)w, (2) 


is the sampling variance of the filtered value at time ¢. The 
sampling error of the filtered value is the difference between w’ y 
and w/ Y,, which is conditional on the values of the true 
series, Y r This is the difference between the filtered value 
obtained from the series of estimates ending at time T and 
the value that would be obtained if that series was observed 
without sampling error. We focus on this component as it is 
the sampling variance that can be altered by changing the 
sample design. The variance associated with Y, has not 
been taken into account. Wolter and Monsour (1981) dis- 
cussed the issue of total variance versus sampling error 
variance. There may be advantages in considering the total 
variance in interpreting the resulting series but when we are 
considering sample design issues, such as the choice of 
rotation pattern, we focus on the component that is directly 
affected by decisions made about the sample design. If the 
sampling error does not contribute significantly to the vari- 
ability of the series then decisions about the sample design 
are not as important as they are when the sampling error is a 
major contributor, although it still seems sensible to use as 
effective a sample design as possible. 

To determine the effect of different rotation patterns on 
the sampling variance of a particular filtered series, we need 
an estimate of V(y,|¥,) for different rotation patterns. 
Previous work on estimating variances of seasonally adjusted 
series has either ignored the rotation pattern and assumed 
independent samples at each time point, or taken it as fixed 
and used an estimation method that takes it into account. We 
need a model for V(y,,|¥) that reflects the effect of the 
different rotation patterns that could be used. 

The analysis of the effect of different rotation patterns is 
simplified if the series of sampling errors has a stable auto- 
correlation structure. The precise form of the autocorrelation 
function will depend on the series and should reflect the 
complexities of the design. For example Steel and DeMel 
(1988) suggested a model for the Australian Monthly Labour 
Force data and Bell and Wilcox (1993) suggested a model 
for the United States Retail Trade series. Bell and Hillmer 
(1990) and Miazaki and Dorea (1993) also considered 
modelling of survey errors by time series models. Dempster 
and Hwang (1993) and Lee (1990) considered approaches to 
estimating and modelling sampling error correlations for the 
S'CPS. 

Our approach is to assume that the series of sampling 
errors, e, has constant variance. A model is needed for the 
correlation between the sampling errors of y, and y,,,. All 
the rotation patterns considered imply that the sample at any 
particular time will consist of a number of panels. A panel is 
a set of units that are included and removed from the survey 
at the same time. When a panel is rotated out of the survey it 
will be replaced by another panel. The set of panels related 
in this way is referred to as a rotation group. Most MLFSs 
use multistage sampling and when.a panel is rotated out of 
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the survey it is replaced by another panel of nearby house- 
holds (see ABS 1992; Singh et al. 1990). Hence it is as- 
sumed that the sampling correlation between estimates ob- 
tained from the same rotation group s periods apart is r(s) 
if no rotation has occurred and d(s) if rotation has occurred. 
We will assume that the estimate at time ¢ is, at least ap- 
proximately, the average of estimates from each rotation 
group and that estimates from different rotation groups, 
which will usually be in different PSUs and spatially well 
separated, are independent. 

These assumptions imply that the sampling correlation 
between y, and y,,, is 


R(s) = d(s) + k(s)(r(s) - d(s)) (3) 


where k(s) is proportion of the sample in common between 
the two time periods. The sample overlap factor k(s) is 
determined by the rotation pattern. For example, for an 
in-for-m rotation pattern k(s) = 1 - s/m,s =0,...,m - 1 and 
zero otherwise, assuming that the same number of dwel- 
lings are added and dropped from the sample each month. 
If different panels in the same rotation group are indepen- 
dent, then d(s) = 0, but in general this will not be the case. 
This model is essentially the same as derived by Scott, 
Smith and Jones (1977). An example of an in-for-4 rotation 
pattern over an eight month period is illustrated in Table 1. 
Different panels are denoted by different letters and the 
subscript indicates the number of times the panel has been 
included in the survey up to the time period indicated. 


Table 1 
Structure of in-for-4 Rotation Pattern 
Time Period 
Reider EPID C2) PBR Ames St 6) Brs37) 
Group 
1 a, a, a, a, 5, b, b, b, 
2 C; d, d, d; d, e; e, e; 
3 Ss SIs &1 & &3 & h, h, 
4 up) ly Ji Jr J3 J4 ky 


In this case (2) is the correlation arising from say a, and 
a,, whereas d(2) is the correlation associated with a, and 
b,. Binder and Hidiroglou (1988) and Fuller et al. (1992) 
provided discussions of the data structure implied by some 
other rotation patterns. 

The assumption that the variance of the sampling error 
series is constant implies that no major changes to the 
sample design or the population structure occur, at least 
over the effective length of the filters being considered. 
The assumption of stable autocorrelations, r(s) and d(s), for 
the population correlation also implies no major changes to 
the sample design or population. Estimates for r(s) and d(s) 
in (3) were obtained from a study by Bell (1998). The 
values used are from the Australian Labour Force Survey 
(ALFS) for the proportion of persons employed and also the 
proportion of persons unemployed and are shown in 
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Table 2. These were obtained by treating the rotation groups 
in the ALFS as replicates and measuring the autocorrelation 
at the rotation group level. A model given in Bell (1998) was 
used to extrapolate values beyond the given lags. 


Table 2 
Autocorelations — ALFS 


Proportion of employed persons 


lag 1 2 3 4 5 6 7 8 
rs) 0.80 071 064 O57 050 045 040 0.36 
Als) SA OMS OFS Fe OA OFS ae ON 2a ON ey Oo1 ee OSLO: 
Proportion of unemployed persons 
1 2 3 4 5 6 7 8 
ns) (0162-10527 3 0:445 0:37) 99 0:31 0260022019 
AS) ODL OLE SOON 0.09 O09 008i O08 0:07, 


Sutcliffe and Lee (1995) studied the standard errors of 
seasonally adjusted and trend estimates of level and move- 
ment under a small number of different rotation patterns. 
They assumed a simple geometric decay model for the cor- 
relations between survey estimates with a population corre- 
lation of p = 0.8, i.e., R(s) = p*, which decreases more rap- 
idly than the values given in Table 2. 


4. LINEAR APPROXIMATIONS FOR 
SEASONALLY ADJUSTED AND 
TREND ESTIMATES 


The X11 method consists of an iterative application of 
moving averages resulting in a symmetric filter for the cen- 
tral values, and asymmetric filters for the values at the begin- 
ning and end of the series. The final seasonally adjusted and 
trend estimates produced by X11 can be approximated by 
linear filters. Several authors; for example, Young (1968), 
Cleveland and Tiao (1976), Wallis (1982), and Sutcliffe 
(1993), have produced linear approximations to the X11 pro- 
cedure. The X11 ARIMA procedure (Dagum 1980, 1988) is 
an extension of X11 and extrapolates the original series at 
both ends by an ARIMA model. The effect of the ARIMA 
extrapolation can be incorporated into the filter weights and 
these weights can be applied to the data alone. Dagum, 
Chhab and Chiu (1996) considered a Cascade method ap- 
proach, where the Cascade filters are a result of the convo- 
lution of the various predetermined linear filters used within 
both X11 and X11ARIMA. We used this approach to real- 
istically approximate both the X11 and X11ARIMA pro- 
cedures. 

Define the matrix whose rows contain the filter weights of 
13 term HMAs for both symmetric and asymmetric filters as H.,,. 

The matrix of weights corresponding to a 3x3 moving 
average (ma) is denoted as S, ., and that corresponding toa 3 x 5 
ma is denoted as S,,,. These are used for estimation of 
seasonal factors. The matrix D is defined as a 12 term 
centered ma and I is an identity matrix. The notation c indi- 
cates the complement of a filter, for example D‘° = I - D. 
The Seasonal Adjustment Cascade filters are written as 


McLaren and Steel: The Impact of Different Rotation Patterns on the Sampling Variance 


S =1-D°S,, .fH,,(D°S,,,D°)I°. 


3x5 3x3 


The trend Cascade filters used for the estimation of trend 
are then found by multiplying the seasonally adjusted filter 
by a trend filter. At the end of the series the Cascade filters 
for trend and seasonally adjusted estimates will differ ac- 
cording to whether X11 or X11 ARIMA is used. 

We consider the following different combinations of the 
internal filters of X11 and X11ARIMA: 


1. Standard X11 Cascade filter: This corresponds to a 13 
term HMA for estimation of trend (H13), 3 x 3 ma for 
the first estimation of the seasonal factors (S1 
3x5 ma for estimation of seasonal factors (S2 
and no modification for outliers. 


2. Standard X11 Cascade filter with ARIMA forecasts: 
This corresponds to use of a H13, S1,,,, S2,,5, and 
extended forecasts from an ARIMA model of the form 
(1 - B)(1 - B'*)y, = (1 - 0.4B)(1 - 0.6B'*)a,, where 
B is the backward shift operator and a, is a white noise 
process, and no modification for outliers. 


3. Short X11 Cascade filter with ARIMA forecasts: This 
corresponds to use of a H9, S1,,,, S2,.., and 
extended forecasts from a model of the form 
(1-B)(1-B")y,=(1-0.3B)(1-0.3B"")a,, and no 
modification for outliers. 


4. Long X11 Cascade filter with ARIMA forecasts: This 
corresponds to use of a H23, S1,,., S2,.5, and 
extended forecasts, from a model of the form (1 - B) 
(1 - B'?)y, = (1 - 0.8B)(1 - 0.88 '?)a,,and no modi- 
fication for outliers. 


3x3)? 
3x5)» 


Combinations 2 and 3 have been observed by Dagum 
(1983) to be applicable in a number of cases. The linear 
approximations chosen allow us to examine the effect of 
different rotation patterns for a range of filters used in 
practice, which involve HMAs of different lengths. 

For each combination of filters the corresponding Cas- 
cade filter provides a vector of filter weights for the sea- 
sonally adjusted estimates and a different weight vector for 
the final trend estimates. These can then be substituted into 
equation (2) to obtain the sampling variances for a parti- 
cular rotation pattern by using the appropriate values for 
V(y,|Y,). When computing change estimates the data 
vector y, remains unchanged and the weights that are 
applied change. For example, w,,, - w, can be used for a 
one month difference. This basic approach is the same as 
that adopted by Wolter and Monsour (1981) who proposed 
estimating the variance of seasonally adjusted estimates 
using (2) with weights chosen that reasonably approximate 
the seasonal adjustment process and using a survey based 
estimate of V(y,|Y). We also consider trend filters and 
different realisations of X11 ARIMA and rotation patterns. 

The X11ARIMA models considered in this paper are 
representative of those commonly used in practice. 


Survey Methodology, December 2000 167 


Additional complications arise from the use of ARIMA Table3 

forecasts in the X11ARIMA approach. For example, we Ratio of the Sampling Variance for Chosen Rotation Patterns 
assume no misspecification of the ARIMA model. The Divided by the Sampling Variance for an Independent Design 
ARIMA model is typically identified and estimated using (Combination 1) 

previous survey data. The sampling error for previous time Rotation SA, SA,,, - SA, iP Pict. 


points could influence the choice of ARIMA model and X11 Pattern emp _unemp emp unemp emp unemp emp unemp 


filters. This could be taken into consideration by modifi- complete 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 
cation of the variance in (2). 1-2-1(5) 0.99 0.99 0.99 1.00 0.99 1.00 0.68 0.79 


vos : ; 1-2-1(8) 0.98 0.99 0.97 0.99 0.98 0.99 0.64 0.77 
initial trend and seasonally a 
The initial tre seasonally adjusted estimates for 1146) 101 101 100 100 117 114 07 082 


time t will be made using the time series of estimates ending 2-2-2(8) 1.02 1.02 061 071 126 123 083 0.95 
at time #, that is y,, giving the filtered value wiy,. The value —2.10-2(4) 1.04 1.04 061 071 135 130 132 1.26 
that would be obtained if there was no sampling error is 3-3-3(6) 1.07 1.06 048 061 1.52 144 1.29 1.25 
w/Y,. The sampling error considered in this paper is 48-48) 1.10 1.08 042 057 169 1.57 1.42 1.34 
wy, — w_Y,. As estimates are added to the series the filtered © 6-612) 1.10 1.08 0.36 0.52 1.76 1.64 1.22 1.22 
value for time ¢ may change, but there will come a time sie : ee ce - nee ‘ie ee ee rae 
peit, at coe ee SOs is ee ee norotation 1.08 1.08 0.24 0.44 1.80 1.69 0.75 0.95 
nmcmiineredwame tontiine poased Onahe' survey cstinates <r  S 
can be written as w*’y,., for a final symmetric weight 
vector w,”. Similarly the final value that would be obtained 


if there were no sampling error would be w*’ Y_, .. Bell and : I Rable 
Kramer (1999) considered the difference w’y, - w*’ Y Ratio of the Sampling Variance for Chosen Rotation Patterns 
oe ate 1 fetes? Divided by the Sampling Variance for an Independent Design 
which includes the forecast error. This difference can be (Combination 2) 
decomposed as Rotation SA, SA,,; ~ SA, T, TT, 
j fe ' F f ae Pattern emp unemp emp unemp emp unemp emp unemp 
WY Wy Yi = (WY, — W,Y,) +0, Y,- Ww", Y,,,)- complete 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 


1-2-1(5) 1.01 1.01 0.99 1.00 1.06 1.05 0.69 0.80 
We have considered how different rotation patterns affect —_1-2-1(8) 1.00 1.00 0.96 0.99 1.07 1.05 0.66 0.78 
the first term in this decomposition. The second term in- _1-1-1(6) 1.045 91203') 1200 1.00), 12205117 9 10-657510.77, 
volves the series observed without sampling error and is = 2-2-2(8) 1.05 1.04 060 0.71 1.32 1.26 0.81 0.92 
unaffected by the sample design, including the rotation pat- 2-10-24), :1.02, 1.03, 0.60 0.71 1.26 1.23) 1.19 1.17 
tern. Bell and Kramer (1999) considered the series of US 3-3-3(6) ~— 1081.06 0.49 0.61 149 140 1.19 1.16 
Housing Starts involving five or more units and showed that Sah 108 5 206, OA 098 sont SO ae pete 
the total variance of the trend series showed large increases are a oe ae pat ae te oa ae 
at the end of the series due to forecasting errors. This is due ee iiineios 0500s or sea et Man 60; 
to the revisions in the initial trend estimates that are made aS) rotation 1.14. 111 024 043. 189 173 059 0.78 
Eomaatestare aaced tothe series, steel and McLaren (2000) °F fr ns ne 
considered the effect of different rotation patterns on the 
observed revision of the initial trend estimates, which is 


, */ . : f Table 5 
wiy,-w* y,,,- They noted that the relative importance o 


Ratio of the Sampling Variance for Chosen Rotation Patterns 


the component due to sampling error will depend on how the Divided by the Sampling Variance for an Independent Design 
true series is evolving around the period being considered. (Combination 3) 
ae 
Rotation SA, SA,,; - SA, il ihe “ if 
Pattern emp unemp emp unemp emp unemp emp unemp 
5. RESULTS complete 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 


1-2-1(45) 0.99 0.99 0.96 0.98 0.99 0.99 0.68 0.79 
We use filters corresponding to the level and one month ~='1-2-1(8)_— (0.97 (0.99 0.93 0.97 0.98 0.99 0.64 0.77 
difference for both the seasonally adjusted and trend esti- 1-1-1) 1.04 1.02, 0.99 0.99 1.11 1.08 0.6 0.72 
mates at the very end of the series. Tables 3 to6 summarise 2-2-208)_-—1.07_ 1.06 0.60 0.71 1.23 1.19 0.89 0.95 
the effect of different rotation patterns for each Cascade filter 2-10-24) = 1.05 1.06 0.61 0.72 1.21 1.20 1.07 1.08 
He we ; 33:36), AS) 1:12 80.515063 a4 1 32 Wal? 02 
combination. These tables give, for a selection of rotation nee rey AC ORY ET TICE oe EA 
patterns, the ratio of the sampling variance of the estimates 66-612) 114 113 037 053 147 139 069 0.82 
under consideration divided by the sampling variance that, Gore 116 113 038 053° 149 140 070 Ost 
would be obtained when there is complete rotation each —j,,_ jg 117. 1.14 034 051 1.52 1.42 061 0.76 
month. The ratios obtained in the middle of the series give —porotation 1.22 1.17 0.25 044 162 150 0.44 0.64 
the same general conclusions (McLaren 1999). let = Fa a eee EaRAS eh) cack eevee 
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Table 6 
Ratio of the Sampling Variance for Chosen Rotation Patterns 
Divided by the Sampling Variance for an Independent Design 
(Combination 4) 
Rotation SA, SA, cate SA, f, 
Pattern 


Ty; ie r, 
emp unemp emp unemp emp unemp emp unemp 
complete 1.0089 009 1200) 100 10059 1008 = 120055 1.00 
1-2-1(5) T0202 0:99 SCOR te 2 5m lel Oe O.7 Seen: 817 
1-2-1(8) 102 | OZR 0.97 0:99 Sas1E2S mane OF O84. 
1-1-1(6) LOG 104 S00 F100 49 Sets oe 092 Se a1eOl 
2-2-2(8) 1 OG 1104 Fe OGOm Oe Same E47 OS Same Ik O9 
2-10-2(4) NGO) WUD OKO) OND; ESO) Needy TAR) ilesi7/ 
3-3-3(6) LOT, LOSS OAS FOG 1:64 ss 1:34 36 
4-8-4(8) MOSt= 104 OMT OSG 7S lk63: 925169 
6-6-6(12) OSes L0G NO SSm Ol 12 OOM ESA Ray 128/268 
in-for-6 109 OT OSS) 05222008 F184 1ESON IE O 
in-for-8 Pe 108 y 1032 0/492 1S) le oGre E73 al 62 
no rotation, Eli) 112s 10:24 043) 2 Or 2.27 ae lee eS 


5.1 X11-— Concurrent Standard Cascade Filters 


The results using the standard X11 filters (combination 1) 
are shown in Table 3. Figures 1(a) to 1(d) show the sampling 
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variance of the level and one month difference for the 
seasonally adjusted and trend estimates at the end of the 
series divided by the variance of the original estimate of 
level plotted against the total number of times a selected 
unit is included. Results for the variable employment have 
been plotted for selected a-b-a(m) patterns and the in-for-m 
rotation patterns for m going from 1 to 30. An in-for-30 
rotation pattern is indicative of having no rotation. 

Columns | and 2 in Table 3 show that for the variance of 
the seasonally adjusted level estimates, rotation patterns 
with no monthly overlap perform well. Using rotation pat- 
terns with annual overlap did not help appreciably. How- 
ever, for the one month change in seasonally adjusted esti- 
mates, the benefit of having high monthly overlap becomes 
evident (see Figure 1(b) and columns 3 and 4 of Table 3). 
The variances associated with the in-for-m rotation patterns 
are effectively a function of 1/m, the proportion of the 
sample that does not overlap. Those rotation patterns used 
in Canada and Australia perform well. The best option is no 
rotation but, as discussed in section 2, this is not a practical 
option. Figures 1(a) and 1(b) show that rotation patterns 
that have the same degree of monthly sample overlap have 
similar variances for estimates of the level and one month 
change in the seasonally adjusted series. 
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Figure 1. Ratio of the sampling variance to the variance of the original series for chosen rotation patterns for combination 1 (X11) for the 
variable employment where A = 4-8-4(8), B = 2-10-2(4), C = 2-2-2(8), D = 1-2-1(5), H = 1-2-1(8), J = 1-1-1(6). 
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For the level of trend estimates the variance increases as 
the amount of monthly sample overlap increases (see Figure 
1(c) and columns 5 and 6 of Table 3). For the in-for-m 
rotation patterns there is a rapid increase in variance as m 
goes from 1 to 5. The rotation patterns of 1-2-1(5) and 
1-2-1(8) perform as well as having an independent sample 
each month and considerably better than rotation patterns 
that involve monthly overlap. This is primarily due to the 
fact that for a moving average, it is better to average over 
independent observations than positively correlated ones. 
The larger variance of the 1-1-1(6) pattern compared with 
that of 1-2-1(5) and 1-2-1(8) suggest that, for those patterns 
with no monthly overlap, the interval between the re- 
inclusion of units in the sample has some effect. 

Figure 1(d) and columns 7 and 8 of Table 3, show that 
for one month changes in trend estimates the variance in- 
creases very rapidly as m increases from 1 to 3 and de- 
creases rapidly as m increases from 4. The in-for-3 rotation 
pattern seems to be the worst among those considered, and 
the currently used rotation patterns can be significantly im- 
proved upon. For example, using a 1-2-1(8) instead of a 
4-8-4(8) rotation pattern would reduce the variance in the 
one month change in trend estimates for employment by 55 
percent and 43 percent for unemployment. While the degree 
of monthly overlap is still a key factor, the pattern of 
inclusion also plays a role, for example the 2-2-2(8) pattern 
has lower variance than the in-for-2 or 2-10-2(4) patterns. 
Moreover, for one month changes in the trend estimates the 
best performing rotation patterns are 1-2-1(5) and 1-2-1(8) 
which perform considerably better than using complete 
rotation each month. This result arises because one month 
changes in trend estimates effectively look at differences in 
the seasonally adjusted series a few months apart and the 
1-2-1(m) rotation patterns lead to positive correlations be- 
tween estimates 3 months apart. Similar results were ob- 
tained in a study by McLaren and Steel (1997) using 
Sutcliffe’s (1993) approximation to X11. 

The results show that for the estimation of the current 
level of trend and the latest movement in trend, the 1-2-1(m) 
rotation patterns give considerably lower sampling vari- 
ances than the rotation patterns currently in use. 


5.2 X11ARIMA —- Concurrent Cascade Filters with 
Extrapolations 


Results for the filter combinations 2, 3 and 4 are given in 
Tables 4, 5 and 6 respectively. Figures 2(a) to 2(d) present 
results for combination 4 for employment. 

Columns 1 and 2 of Tables 4, 5 and 6 show that rotation 
patterns with low monthly overlap perform almost as well 
as complete rotation for seasonally adjusted level estimates. 
Rotation patterns with high monthly overlap have higher 
variances, particularly for combination 3 which corresponds 
to the use of the 9 term HMA. 

There is minimal difference between the ratios of the 
four different combinations for the one month change in the 
seasonally adjusted estimates (columns 3 and 4 in all 
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tables). Rotation patterns with high monthly sample overlap 
still perform better than those with low or no monthly over- 
lap regardless of the X11/X11ARIMA combination used. 

For the level of trend estimates, rotation patterns with a 
higher degree of sample overlap again have a greater va- 
riance ratio. The 1-2-1(5) and 1-2-1(8) rotation patterns still 
out-perform the other rotation patterns for each combination 
of filters, although they do not perform as well as an inde- 
pendent sample for combinations 2 and 4. 

For one month changes in the trend estimates the better 
performing rotation patterns are again 1-2-1(5) and 1-2-1(8) 
which perform better than the independent sample for all 
four combinations of filters. For combination 3, rotation 
patterns with high monthly overlap perform equally as well 
as the 1-2-1(7) rotation patterns. For combinations 2 and 3 
the 1-1-1(6) pattern is slightly better than the 1-2-1(m) 
patterns. Substantial improvements over the currently used 
rotation patterns can be achieved by using 1-2-1(m) rotation 
patterns. For example, for the employment variable, 
changing from an 4-8-4(8) to a 1-2-1(8) would produce 
gains of 42, 25 and 64 percent using combinations 2, 3 and 
4, respectively. 

These results are based on the ALFS correlation esti- 
mates which, being based on survey estimates, will be 
subject to sampling error. The trend filters considered are 
not derived using these estimates. The same general conclu- 
sions concerning the impact of different rotation patterns 
are obtained for the two correlation models which use rea- 
sonably different correlations. We believe that the conclu- 
sions will apply for the range of correlation models con- 
tained between these two models. Similar conclusions are 
also obtained by McLaren and Steel (1997) using a corre- 
lation model derived by Steel (1996) for UK employment 
and unemployment. 


6. DISCUSSION 


The rotation patterns currently used, such as in-for-8, 
in-for-6 and 4-8-4(8), are sensible if the one month change 
in seasonally adjusted estimates are the key statistics to be 
analysed. We believe that examination of the one month 
change in seasonally adjusted estimates is often not a 
reliable way of assessing current trends. It is necessary to 
look at the pattern of change over recent months. This can 
be done using filters to obtain an estimate of the trend. The 
results here suggest if the main use of the survey is to 
provide an assessment of trend then quite different rotation 
patterns should be used. Specifically, the 1-2-1(m) rotation 
patterns performed well for reducing the variance of the 
level of trend estimates and the difference between two 
consecutive trend estimates for a range of different filter 
combinations. The 1-2-1(m) rotation patterns also per- 
formed well for the sampling variance of the seasonally 
adjusted level estimates. Hence, in designing the rotation 
pattern for a repeated survey, the relative importance of 
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Figure 2. Ratio of the sampling variance to the variance of the original series for chosen rotation patterns for Combination 4 (X11 ARIMA) 
for the variable employment where A = 4-8-4, B=2-10-2(4), C=2-2-2(8), D=1,2-1(5), H=1-2-1(8), J=1-1-1(6). 


seasonally adjusted and trend estimates needs to be care- 
fully considered. Examining Figures 1 and 2 shows that the 
rotation pattern 2-2-2(8), is a reasonable compromise if the 
level and one months change in seasonally adjusted and 
trend estimates are both considered important. Bell (1999) 
also considered the effect of four different rotation patterns 
on the sampling variance of the level and one month change 
in the original, unadjusted, estimates and also trend esti- 
mates obtained using X11 and a 13 point HMA. He also 
identifies the 2-2-2(8) rotation pattern as a compromise 
design. 

Even if analysts do not formally use trend estimates, the 
assessment of trend will involve looking at changes in 
seasonally adjusted estimates a few months apart. McLaren 
(1999) gives results which show that the 1-2-1(m) rotation 
patterns will be suitable if the assessment of trends involve 
looking at changes in seasonally adjusted estimates over 3 
or 6 months. The results also suggest that such rotation 
patterns perform well for estimates of the change in trend 
estimates over the most recent 3 and 6 months. 


The evaluation criterion used in this paper is the sam- 
pling variance of the trend and seasonally adjusted esti- 
mates, which is the factor affected by the sample design. 
Steel and McLaren (2000) considered assessing different 
rotation patterns in terms of the degree of revisions of these 
estimates at the end points and reached similar conclusions 
regarding the rotation patterns. 
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Hierarchical Bayes Estimation of Small Area 
Means Using Multi-Level Models 


YONG YOU and J.N.K. RAO! 


ABSTRACT 


Standard multi-level models with random regression parameters are considered for small area estimation. We also extend 
the models by allowing unequal error variances or by assuming random effect models for both regression parameters and 
error variances. We present these models in a hierarchical Bayes framework and estimate a small area mean by its posterior 
mean. Posterior variance of the small area mean is used as a measure of precision of the estimate. It automatically takes into 
account the extra uncertainty associated with the hyperparameters in the multi-level model. Gibbs sampling is used to 
compute the posterior means and posterior variances of small area means. Rao-Blackwellized estimators that reduce the 
Monte Carlo errors are obtained. Bayesian model selection and sensitivity analysis are also studied. The procedure is 
illustrated using data on household income in some counties (small areas) of Brazil. 


KEY WORDS: Gibbs sampling; Hierarchical Bayes; Multi-level model; Sampling error variance; Small area. 


1. INTRODUCTION 


Small area estimation has received a lot of attention in 
recent years due to growing demand for reliable small area 
estimators. Traditional area-specific direct estimators do not 
provide adequate precision because sample sizes in small 
areas are seldom large enough. This makes it necessary to 
employ indirect estimators that borrow strength from 
related areas; in particular, model-based indirect estimators. 
Battese, Harter and Fuller (1988) proposed and applied a 
nested error regression model to provide model-based small 
area estimates. The model takes the form 


if ; ; 
Vy = Xj Bt+v, +e, =1,...051=1,...m, (1) 


where yj, are the observations associated with the sampled 
units in the i-th small area, i = 1, ..., m,x,. 1s the p x 1 vector 
of unit-level explanatory variables, B is a set of p fixed 
regression parameters, Vp, a independent area effects with 
E(v);) =90 and V(v;) =6,. The e,.’s are assumed to be 
Bee gent random error variables with E(e,) = 0 and 
AG i) = oO . Vo; and e,, are also assumed to be piescadede 
For the whole population, model (1) applies with 2, 
replaced by N,, the small area population size. The model 
(1) may be expressed in matrix notation as follows 
Y, =X, B + vo, ls Hert = 1 x.,7N; 

eC ag Dye X= (Xjp-0X,,) isan,xp matrix, 

=(1,..,1)7 is the unit vector “Of length n,, and 

"2 (e,, a Seka 

Holt and Moura (1993) extended the above framework 
to a multi-level model by introducing random regression 
coefficients and then relating them to area-level explanatory 


variables to explain some of the between small area 
variation. The model can be stated as follows: 


Y,=X,B,+e,,B,=Z,y +, (2) 


where Z, is the p x q design matrix of area-level variables, y 
isaqx 1 vector of fixed coefficients, and Vi= (Vices Y, De 
is a px 1 vector of random effects for the i-th area. The 
v,’S are independent across areas and have a joint 
distribution within each area with E(v,) = 0 and V(v,) =, 
where the variance covariance matrix ® is unknown. Note 
that model (2) effectively integrates the use of unit-level 
and area-level covariates into a single model. Holt and 
Moura (1993) and Moura and Holt (1999) extended Prasad 
and Rao’s (1990) framework to the above multi-level model 
to get the best linear junbiased predictor (BLUP) of the 
small area mean pl, =X 'B, assuming that N, is large, where a 
is the px 1 vector of known population means of the 
auxiliary variables for the i-th small area. They also 
obtained the empirical BLUP (EBLUP) and a second order 
approximation to the mean squared error (MSE) of EBLUP 
for the multi-level model. Using household income data in 
some counties (small areas) of Brazil, they demonstrated 
gain in efficiency of the EBLUP estimators over the 
EBLUP estimators obtained from nested error regression 
models. Ghosh and Rao (1994) and Rao (1999) provide a 
detailed overview of model-based methods for small area 
estimation. 

In this paper, we study the multi-level model (2) in a 
hierarchical Bayes framework and extend the model to 
more general multi-level models which allow fixed unequal 
error variances or random error variances. The small area 
mean J, is estimated by its posterior mean and its precision 
is measured by its posterior variance. Posterior variance 
automatically takes into account the extra uncertainty 
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associated with the hyperparameters in the multi-level 
model. We use the Gibbs sampling method to compute the 
hierarchical Bayes estimates and the associated posterior 
variances. Section 2 presents the hierarchical Bayes 
multi-level models with different assumptions on error 
variances and related Gibbs sampling inference. Section 3 
illustrates our methodology and studies model selection and 
sensitivity analysis by employing data on household 
incomes in some counties (small areas) of Brazil. And 
finally in section 4, we give some comments and 
concluding remarks. 


2. MULTI-LEVEL MODELS AND GIBBS 
SAMPLING INFERENCE 


2.1 Equal Error Variances 


We consider a hierarchical Bayes representation of the 
multi-level model (2) as follows: 


Model 1: 


(i) | Conditional on B, and o., Yi ’s are independent with 
Vij | B,, o, ~N(x; BOs ») 


=i... @) 


A aod PRED B 
(ii) | Conditional on y and ®, B,’s are independent with 
Bly, P~N,(Z,y, ®), (i = 1, ....m). (4) 


To complete our Bayesian model specification, we adopt 
the prior distributions for parameters as follows: 


(iii) Marginal prior distributions:  y ~ N, (0, D), 
te ae: b) and Q~ W (a, R), where tL =O, 
Oe ®!, and D,a,b,a and R are known. 


In step (ili) of Model 1, G(a,b) denotes a gamma distri- 
bution with density given by f(x) =b%/I(a)x71e ™, 
a>0O,b>0,x>0, and W, (a, R) is a Wishart diceribution 
with density function 

a 


[R|? 
ag Te) 
2 
where X>0,R>O and IT AC) is multivariate gamma 
function defined as 
P(p-1) 
l(a) =m 4 


a-p-l 
[xy 


f(X) = exp{-~ tr (RX)}, 


raT{a+da -p). 


Remark 1.1: The prior distributions in step (iii) are conju- 
gate with the sampling and population distributions given 
by (3) and (4) in the sense that they lead to full conditional 
distributions for y, t, and Q that are again normal, gamma 
and Wishart distribution, respectively. The Wishart distri- 
bution is the multivariate version of gamma distribution for 


the inverse variance covariance matrix of random effects. 
The importance of conjugacy may be exploited as follows: 
(1) In the Gibbs sampling step, without conjugacy the full 
conditional distribution for any parameter will be known up 
to normalizing constants. In this case, more sophisticated 
random generation will be required. (2) Closed-form full 
conditional distributions may be employed to find the 
Rao-Blackwellized estimators of the posterior means and 
posterior variances, and thus to improve posterior estima- 
tion. In general, for Bayesian inference, choosing priors is 
not a simple job because any proper prior on the model 
parameters is a plausible candidate. This is a limitation of 
Bayesian methods. 


Remark 1.2: It is important to note that we have used 
proper priors on all the unknown parameters to ensure that 
all the posterior distributions are proper (Hobert and Casella 
1996). Hence we do not face the problem of some poste- 
riors being improper. Values for the parameters of the 
priors (i.e., hyperparameters) are chosen to reflect a fairly 
vague knowledge of the prior distributions. Details will be 
given in section 3 on data analysis. 


Remark 1.3: In Model 1, we assume equal error variance o 
for all small areas. In practice, however, variances of 
sampling error could be different for different small areas. 
A more general model should allow possibly different error 
variances. In sections 2.2 and 2.3, we will introduce 
unequal error variance and random error variance models. 

We are interested in finding the posterior distributions of 
B,’s given the data Y = ({y,}, 7 = 1,.. lity f= las 1) AO 
in particular in finding the posterior estimates of small area 
means H, =X; B,, which depend on the estimates of B.. 
Direct evaluation of the joint posterior distribution involves 
high-dimensional numerical integration, and is not compu- 
tationally feasible. Therefore, we use the Gibbs sampling 
method (Gelfand and Smith 1990) to generate samples from 
the joint posterior distributions. To implement the Gibbs 
sampling under Model 1, we need the full conditional 
distributions given ae 


(i) Fomn—ee. 
[B,|¥, 7, 2,0) "SN ((4,X,'X, “ayy 1 
(1,X,'Y, + OZ), (4,X/'X, + Q)") 


erin 


Gi) EW1¥, BQ x1-N,((E" 2,02, + D-1)" 
(pr'iz7op, (9% Zaz, 4D") 

(ii) [Q|¥,B,y.t] ~ , 
[a+ma+ 257 (B, - ZB, - z)") 


: 1 um 
Gy) (eB arG[ a+ LEP, 


b+ (DM, -X,B), -X,B). 
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Since all the full conditional distributions have 
closed-form, it is easy to generate samples. Gibbs sampling 
cae is as, follows: (a) Using starting values y, Q® and 

draw Bi”, i = 1,...,m, ee y, Q, T,]; (0) Draw y? 
A [y|¥, B, Q, 1] asneipC: a ,m, a and 1; (c) 
Draw 2” from [Qly B, ane re BY? i=1,....m, ¥ 
and t, ; (d) Draw oe from [t,|Y, B, y, Q] using Bo, 
i=1,...,m,yY and Q®, Steps (a)-(d) complete one 
sampling cycle. Perform a large number of cycles, say f, 
gs “burn-in” period, until convergence and then treat 

(po, f=1,..,msy; QC%;c0. k= 1,..,G} as G sam- 
ples from the joint posterior of B, PA, 201,52 ands, 
pubDese a sample of oe G 4s obtained as 
{p?, 7 =1,.., m3; Q®; M; k= 1,...G)}. To obtain an 
estimator of the posterion mean ‘. B, one can use the 
sample mean of the (p* y. Since B, has a closed form full 
conditional distribution, we can use the sample mean of the 
conditional expectations {E[B,|¥, y, Q®, 1 ]} to im- 
prove our estimation, since E(B, \Y )= E(EB,|¥, y,2,1,)), 
and Var(B,| Y)>Var(£(B,| ¥, y, Q, t,)). This modification 
is based on the well-known Rao-Blackwell theorem and the 
corresponding estimator is the so-called Rao-Blackwellized 
estimator (Gelfand and Smith 1990, 1991). Thus we have 
the following two alternative estimators for B;: 


a() CO 
ao lies (5) 
P Gia P, 


and 


G 
a 1 Kk 
piers PTE (BLY, y®, 2%, ) 
k=) 


Ge Loy), 6) 


where Bp” is the Ba estimator and Br (RB) is the Rao- 


Blackwellized estimator. Both Bi ae pee are aoe 
for the posterior mean. However, B\ ) is better than B° 
terms of simulation standurd error (Gelfand and Sinith 
1991). 

The corresponding estimators for the small area mean p, 
are given as 


G 
ieee aap Sree 7) 
Ga 
and 
Ara te 1S yr, OXY + Q®)? 
Ga 
(XTY, + QZ), (8) 
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We anticipate that both Tage and ji PR will give almost the 
same point estimates. However, it will be of interest to 
compute and compare the simulation standard errors of 
these two estimators to evaluate the effects of Rao- 
Blackwellization; see section 3. 

To obtain the posterior variance of y,, we first find the 
posterior variance of B,, since V(u, \Y j= xe VB, |Y)X,. 
Note that 


V(B,|¥)=E(V(B,|¥, ,2,1,)) + V(E(B IY, ¥,2,7,)) 
=E(V(B,|¥, ¥,2,1,)) +E (E(B, ¥, 7,.0,1,) 
- [E(EB,|¥, 7, 2,7,)P. 9) 


Using (9), the Rao-Blackwellized estimator of the posterior 
variance of B., denoted eI V(B, ), can be obtained using the 
Gibbs samples {B\”, i= 1, ..., m;y®; Q®; ©: k=1, ..., G}; 
see Appendix Al. ue posterior variance of small area 
mean iJ, is then estimated by 


Vu) =X; VB)x,. (10) 


The same estimation procedure can be applied to the 
sampling error variance o.. Since conditionally o. has an 
inverse gamma distribution, the Rao- Blackwellized esti- 
mator of the posterior mean of 0. is obtained as 


G 
-2(RB) 1 
oO, Sj 


b+ => Of 2a Gere 2% oD 
=1 j=l 


% 


‘. “1 
[a2 45» -1] (11) 
ore 


Since we are mainly interested in estimating the small area 
means, calculation of the sampling variance is only for the 
purpose of model selection. Details on model selection will 
be given in section 3.2. 


2.2 Unequal Error Variances 


In practice, it is more realistic to allow unequal error 
variances for the sampling errors. Let o be the true 
sampling error variance for the i-th small area. A straight- 
forward extension of Model 1 leads to the following 
hierarchical Bayes multi-level unequal error variance 
model: 


Model 2: 
(i) Conditional on B, and o, yi,’ are independent 
with 
Vij | Pa o, ~N(QX;j Boo i (12) 
(i= i ve 9 mj = F he ae 
(i1) Conditional on y and 9, B,’s are independent with 


Bly, O~N,(Zy, ), (i = 1, ....m); (13) 
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(i11) Serie prior distributions: y ~ IN; (0, D), 


Pes GG@,b), and Q ~ W,(a,R), where 
= 6, *Q= Oo, and D,a,b,a and R are 
Le ibe 


Remark 2.1: Model 2 reduces to Model 1 when Oo = o, for 
all i. From a hierarchical Bayes perspective, extension from 
the equal error variance model to the unequal error variance 
model is straightforward. Also there is no difficulty in the 
Gibbs sampling implementation. 


Remark 2.2: t,’s are assumed to be independent and have 
prior distributions G(a,,b,), where a, and 5, are known 
hyperparameters and usually chosen to be very small to 
reflect a vague knowledge about t,’s. 

The full conditional distributions for Gibbs sampling 
under Model 2 are given by: 


(1) Jetje als oe 
[Bl ¥, y, 2, 1] NA(t 2 ay 
(1,X,'Y, + 077), (4x? KO ne 


Ae 


"i = 
fit) N, [$$ 27a2,-p-} 
m m a! 
[$20 ? E2302," 
i=l i=] 
(ili) t]~ W 


em po Sess 
i=1 
(iv) For i =1,. 
area eAGia, +My b, + 
“(¥,- XB)", -XB))- 


3 


For Model 2, the empirical estimators of the posterior 
means of B, and uy, have the same form as (5) and (7). The 
Rao- Blackwellized estimators ee a nde Ware a 
estimators given by (6) and (8) with ae replaced by ro 
Estimator of Posteo variance is Vu, .) given by (10) with if 
replaced by a 

For the purpose of model selection and model compa- 
rison, we also find the Rao-Blackwellized estimator of the 
posterior mean of o under Model 2 as 

G28) - id hes (Ky T (k) 
= GE [bit HBP) BP) 


(a, + =n, -1) ; (14) 


2.3 Random Error Variances 


In Model 2, we assumed unequal error variances for the 
sampling errors. Kleffe and Rao (1992) used a simple 
random error variance model to derive the best linear 


unbiased predictors for small area means. In this section we 
extend their model to the multi-level case. We assume 
random effect models on both regression coefficients B, 
and sampling error variances o, which leads to Model 3 
given below. 


Model 3: 


(1) Same as in Model 2; 

(ii) Same as in Model 2; 

(ii1) Conditional on n and 4, t,’s are independent with 
rind Gqn,d), (15) 


=) 
where T, = 6; ; 


(iv) Marginal prior distributions: y ~ iV (0, D), 
Q ~ W (a, R),n~ U* and A~ U*, where u> 
denotes a uniform distribution over a cubes of R* 
with large but finite length, D,a and R are 
known. 


Remark 3.1: In Model 3, we assume that 1,’s are iid 
gamma random variables with unknown hyperparameters n 
and i. Thus we have population models for both regres- 
sion coefficient B, and sampling variance o.. In Model 1 
and Model 2, we considered modelling B. only and as- 
sumed vague proper prior distributions on 6, or 6;. 


Remark 3.2: Assumption (iii) may not be a good popu- 
lation model for all t,’s. Alternatively, we can model t, in 
a more realistic way, as in the case of B,, by specifying a 
regression model for the logarithm of t,. This may require 
some auxiliary information related to t,. In the data analysis 
of section 3, however, we simply used G(n,A) as the 
population model for t,. Generally it is not easy to model 
the sampling variances when they are unknown. 
The full conditional distributions for Gibbs sampling 
under Model 3 are given by: 
(1) Bondi iit 
[B IY, ty, Qn, A) "8 N,(4,X,"X, + 2)" 
(1,X/7Y, + 227), (1,X,'X, + 9)"} 
(ii) For i = 1,. 
[t,|¥, B, ae a 


7 Asien aas 
Gi) fy, Bonal - N,(("%,270z, +d)" 
(57 ,2,708,), (5%,z,'az, + D>") 
(iv)  [Q|¥, B, 07, 0,4] ~ W, 


f atm, R+ <r (B, 2277 )¢B, 2 


ind © 
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(v) [nl¥, B,t, y,Q, A) (P(n) "a" 7, )" 


iy OTN Baty Os |= G(mn + By iey| 


For Model 3, the posterior estimators of B, and py, have 
the same forms as those given for Model 2. Under Model 3, 
the Rao-Blackwellized estimator of the posterior mean of 
6; is given by 


gre) - 


rise 1 kT 
= pee OX Bi 
z », [MO + (FX 8) 
1 -1 
x(¥,- XPM +>2,- 1). (16) 
Under Model 3, [n|¥, B, t, y,Q,4] is known only up 
to a multiplicative constant. However, since 
In| ¥, B, t, y,Q,2] is a log-concave function of n (see 
Appendix A2), adaptive rejection sampling method of 
Gilks, Best and Tan (1995) can be used in the Gibbs 
sampler to generate samples from the conditional distri- 
bution [n| Y, B, t, y, Q, A]. 


3. DATA ANALYSIS 
3.1 Data and Model Description 


Following Holt and Moura (1993) and Moura and Holt 
(1999), we considered the estimation of household income 
in some counties (small areas) of Brazil. Holt and Moura’s 
original data contains 140 small areas with the sampling 
units taken from each area by simple random sampling. The 
hierarchical Bayes method does not require the number of 
small areas to be large, unlike in the case of EBLUP 
method, for getting standard errors. Therefore, we used 
only a small part of the original data set in our data analysis 
for simple illustration. Our data set contains a subset of 10 
small areas with 28 sampling units obtained by simple 
random sampling in each area. 

Let y,, denote the j-th household’s income in the i-th 
small area. There are two unit level auxiliary variables, 
namely x, and x,, where x, denotes the number of rooms 
in a household and x, denotes the educational attainment of 
Head of Household. The sampling model is given by 


T 
Vij = Xi; By + &i; = Box + X17 Bay + X24; Bay + 2 (17) 


where x,,, denotes the number of rooms in the j-th house- 
hold of small area 7 and Xo denotes the corresponding 
educational attainment of Head of Household. Values of 
x, and x,, are centered around their respective overall 
sample means and e,, is the sampling error variable with its 
distribution specified by the three error variance models 
discussed in section 2. 

In the sampling model (17), B, is the random regression 
coefficient corresponding to the i-th small area and is 
modelled as 
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Bo Yo Yon Pu dia Yuet Vi Bo; = Yoo * Ya12; + Vays 


where y = (Yo Vio Vir Yoo Wei is the unknown vector of 
fixed regression parameters, ViACVAV i. Ve) is the i-th 
small area random effect vector distributed as 
v,~N,(0, ®), and z, is an area level variable defined as the 
average number of cars per household in each small area. 
Value of z, is also centered around its overall sample mean. 

We used the three models discussed in section 2 for our 
data analysis. Vague proper prior distributions on unknown 
parameters are specified as follows: y ~ N,(0,D) where 
D = diag (10*, 10*, 10*, 10%, 10*), thus Yo. Yio. Yy3> Yoor Yor 
are assumed to be independent normal variables with a 
mean of 0 and a standard deviation of 100, so that a 95% 
prior interval is around +200, and the prior will be locally 
uniform over the region supported by the likelihood. 
Alternatively a uniform prior on a suitably wide interval 
could be given, such as U(-200, 200). A Wishart prior 
W, (a, ay is specified for the inverse covariance matrix 
Q’- ®°!. To represent vague prior knowledge, we have 
chosen the degrees of freedom a for this distribution to be 
as small as possible, ie, a=3, the rank of 
(2 (Spiegelhalter, Thomas, Best and Gilks 1996). The scale 
matrix R is specified with diagonal elements equal to 1 and 
off-diagonal elements equal to 0.001, which represents our 
prior guess at the order of magnitude of the covariance 
matrix. For Model 1 and Model 2, a gamma prior G(0.001, 
0.001) is assumed for t, and t,‘s. For Model 3, 
t, ~ G(m,A), and n and 2d are assumed to be indepen- 
dently distributed as U(0,10000), i.e., the uniform distribu- 
tion over a large interval. We anticipate that the vague 
proper priors on the hyperparameters would approximate 
the flat priors reasonably well and thus would have minimal 
effect on the posterior estimation. 


We implemented the Gibbs sampler for the three models 
using the BUGS program (Spiegelhalter et al. 1996), aided 
by CODA Splus function (Best, Cowles and Vines 1996) 
for assessing convergence. The BUGS program constructs 
the necessary full conditional distributions and carries out 
the Gibbs sampling as long as we specify our models using 
the BUGS language. Priors and initial values of the 
parameters must be specified in the program. For each 
model, the Gibbs sampler was first run for a “burn-in” 
period of 2,000 iterations, then 5,000 more iterations were 
run and kept for model analysis and estimation. 

Our interest s to estimate the small area mean 
Hi; NG Bis Bout 4a) Bait Aga: where X,, and_X,, are the 
i-th small area population means of the auxiliary “GIA a 
and x,, respectively. For this, we will first select a model 
for the data set, then we will present the model-based 
estimates for the small area means based on the selected 
model. 


3.2 Model Selection 


We have proposed three models in section 2 based on 
different assumptions on sampling variances. To examine 
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which model fits the data, we first obtained the posterior 
estimates of the sampling variances under the three models. 
We also calculated the ordinary least square (OLS) 
estimates of the sampling variances within each area using 
only the area-specific data. Table 1 shows the Rao- 
Blackwellized estimates of the sampling variances under 
the three models as well as the OLS estimates. 


Table 1 
Estimated Sampling Error Variances 
Area OLS Model 1 Model 2 Model 3 
1 38.17 76.86 40.18 63.60 
Z SRD 76.86 34.24 62.13 
3 81.26 76.86 94.77 79.58 
4 48.73 76.86 52.01 67.27 
5) 115.98 76.86 121.65 87.70 
6 90.74 76.86 94.35 79.78 
7 101.67 76.86 101.67 82.14 
8 135.65 76.86 159.94 97.96 
9 59.10 76.86 63.37 70.57 
10 62.86 76.86 65.72 TDD 


From Table 1, the OLS estimates indicate large varia- 
tions among the ten small areas. Model 1 assumes an equal 
error Aran’ 0. for all areas and o is estimated by 
see = 76.86, which i is much smaller than the OLS esti- 
mates for some areas. Model 2 assumes unequal error va- 
riance oO across areas. Under Model 2, the estimated error 
variances one to some extent show the feature of the 
areas; G, are consistent with the pattern of the OLS 
estimates. The most notable result is ae )=121.65 and 
Ga. = 159.94, which show that there are larger variations 
within small areas 5 and 8. Model 3 assumes o's to be 
random variables distributed as G(n, A). Under Model >, 
Boke tend to be equal to and have moved toward 
= 76.86. The results in Table 1 suggest that Model 2, 
die unequal error variance model, could be the best model 
for our data set. For further investigation, we now present 
a cross-validation study to select a best fit model. 

In order to study how the data support each model, we 
calculated the cross-validation predictive densities for each 
data point y,. The cross-validation density for y,, is the 
conditional density f 0; Xo )), where Y/,. denotes all data 
except Y,. We looked vat ne value of TO, |Y i) at the 
observed data point, the so called conditional predictive 
ordinate, or CPO, for each of the three models. That is 

CPO; u FOV; sl Yip, obs ), 
where y,, ,, denotes the observed data point. Since CPOs 
are nothing but the observed likelihoods, models with larger 
CPOs provide better fit to the observed data. By using the 
output from the Gibbs sampler, we can calculate the CPOs 
for all data points. For example, under Model 1, we have 


fY) 
AYiaw oe 
Fl Yay) 70) 
2 1 
Yin» By © e 2 2 
ae f(B,, 6, |Y)dB,doz 
Ril bron) 
: 1 
a f(B,02|Y)dBdo? 
FV | Yip By © pi 


ale noting ree the, y;,'S are conditionally independent, 
IO; i wr pise:).= f(y, |B) 5 °), the CPO values are 
isiatne 


CPO eae nena! Solaris ase 
oT yo 1 (18) 
Gel bk Uk 
G LOR sais aD) 


where f ( Yi IB,» 02) is the normal density function given by 
(3).\-For Model 2 and Model 3, the CPOs are calculated 
with o2” replaced by o,” in (18). More detailed 
discussion can be found in Gelfand (1995). 

We present a CPO plot for the three models in Figure 1. 
Clearly Model 2 is the best model among the three, because 
a majority of CPO values for Model 2 are significantly 
larger than those for Model 1 and Model 3. Model 3 is 
slightly better than Model 1 in terms of CPO values. Also 
there are small CPO values for all three models, which 
indicate that our model assumptions may not be very well 
satisfied by our data set. 

According to the sampling variance estimates given in 
the Table 1 and the CPO plot, we conclude that Model 2 is 
a good model for our data. Therefore, we used Model 2 to 
find model-based estimates of small area means and 
associated posterior standard errors. 
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Figure 1. Model selection: CPO comparison plot 


3.3 Result of Estimation 


We now present the estimates of the small area means 
based on Model 2 only. Table 2 presents the estimated 
posterior small area means and the corresponding posterior 
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standard errors. Our study found that the ee esti- 
mator m ; ) and the Rao-Blackwellized estimator m nie gave 
almost the same point estimates, aS we only reported the 
estimates obtained by using ii‘ For comparison, we 
also calculated the direct estimates (sample means) and 
corresponding direct standard errors for the ten areas. It is 
clear from Table 2 that the model-based estimates are 
substantially more efficient than the direct estimates. The 
posterior standard errors are much smaller than the direct 
standard errors. 


Table 2 
Estimates of Small Area Means 
Area y, Sc. niger Sie! 
1 11.08 9.53 10.23 0.81 
2 7.91 6.82 9.84 0.85 
5 13.48 14.15 13.01 1.08 
4 6.53 8.01 10.95 i 
5 19.52 14.96 17.87 Sia) 
6 meat 11.38 10.21 0.93 
7 8.72 11.24 9.58 0.97 
8 12.81 13.99 10.30 1.19 
9 10.18 8.76 11.34 1.01 
10 10.01 11.30 9.79 0.87 


In order to study the effects of Rao- DES yg eS we 
calculated the simulation standard errors of ji” ; and ji ; a, 
which are respectively the sample standard errors of 
VG Seay and {.X; TE [BY. 7°, Q®, aLaWe Table 3 presents 
the simulation standard errors. It is clear from Table 3 that 
the Rao-Blackwellized estimator ii‘ i”) has much SITE) 
simulation standard error than the empirical estimator {i ; = 
for all areas. In all cases the standard erOr or w Le ) is about 
50% to 75% of the standard error of ihe ) demonstrating 
the benefit of Ae Blackwellization. Thus ii‘ is more 
stable than ‘Tings ; ) when used to produce point estimates for 
the posterior means in computational Bayesian analysis. It 
ae be mentioned that the simulation standard error of 
ae is also an estimator of the BSA eee error. 
Thus the simulation standard error of ii‘ in Table 3 is 
almost identical to the estimated standard error of ins” in 
Table 2. 


Table 3 
Simulation Standard Errors 
ty CS en eee 


ea - ow; H; 
1 0.817 0.506 
2 0.862 0.498 
3 1.090 0.548 
4 1.101 0.604 
5 1.583 0.878 
6 0.930 0.481 
i 0.978 0.480 
8 1.208 0.842 
S 0.997 0.524 
10 0.869 ODis 
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3.4 Sensitivity Analysis 


In Model 2, the error variances 1, = 6, * are assumed to 
be independent with prior distributions G(a,, b,) or o with 
the inverse gamma IG(a,, b,), where a, and b, are known 
values chosen to reflect our prior knowledge about O.. In 
practice, it is always difficult to obtain accurate information 
about the sampling variances. Also, as the number of small 
areas m increases, the number of variance components o; 
will increase. We are interested in, the possible effects 
caused by the choice of priors on oO ’s; in particular, we 
would like to evaluate the sensitivity of the posterior means 
to the choice of priors on the sampling variances o. In our 
data analysis, a, and b, were chosen to be 0.001. Thus we 
used proper priors with very small parameter values for the 
variance components to reflect our vague knowledge about o.. 
In order to test the sensitivity of the posterior estimates to 
the choice of a, and b, under Model 2, we set a, = b, at six 
different values, i.e., 0.0001, 0.001, 0.01, 0.1, 1, and 10. 
Since 


(1 |¥,B,y,Q) ~ G(a,+ =n, b+ (1, - XB)" 
GX) aca, 


the sample effects n,/2 and (Y, -X,B,)"(¥, -X,B,)/2 
dominate the prior information a, and 6, when a, and b, 
are small. Thus IG(0.0001, 0.0001), IG(0.001, 0.001), and 
IG(0.01,0.01) may be viewed as noninformative priors 
whereas IG(1, 1) and IG(10, 10) may be regarded as 
informative priors. Table 4 presents posterior means under 
Model 2 using the different priors on o., and Table 5 
presents the corresponding posterior variances. 


Table 4 
Comparison of Estimated Small Area Means 

Small IG(a,, 6), a, = 6, 

Area 0 0 0.01 0.1 1 10 
1 1023 510123" 53 10:237 9 10245 1025) 037, 
2 9.84 9.84 9.84 9.83 9.82 9.62 
3 13-00% =- 13:00 = 13:01 = 13:01 -13:07— 13:09 
4 10.95 10.95 10.95 10.95 10.94 10.61 
5 GESCm ye li - 87/85 ei) Oo ly ee OM Oey) 
6 LODO D1 S10 Zi al O21 ODS e028 
7 9.58 9.58 9 9.58 9.63 O51) 
8 10229. S10: SO A103 0 R102 60 LOS 7a 1OI86 
9 11345) 134 oli Sell Slee E32 ee e2S 
10 Me) O79 9.80 9.79 9.82 9.92 


It is clear from Table 4 that the small area mean 
estimates are very stable: they are not sensitive to the choice 
of a, and b.. However, as shown in Table 5, the posterior 
variances decrease as the priors on 6, become more 
informative, and lead to smaller coefficients of variation 
(CV). This indicates that we can improve estimation results 
for small areas in terms of CV if we have more prior 
information on the sampling error variances. In our study, 
we only considered the case a, = b,. A more extensive study 
would involve different combinations of a, and b.. 
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Table 5 
Comparison of Estimated Posterior Variances 

Small IG(a,, b,), a; = b; 

Area 0 0 0.01 0.1 1 10 
1 0.658 0.658 0.658 0.656 0.653 0.499 
2 0.724 0.724 0.724 0.711 0.684 0.462 
3 ly) ii Tela SIS OSONT/ 
4 1220) Sle2207 S21 See e202 OL919 
5 DIAS Se 45S 44 42a Omens 
6 0.871 0.870 0.870 0.830 0.826 0.699 
1 01933) 0!9335 0931) = 0930 0914 O79 
8 NGS TG? TAOS ESI) LIS pI 1.337 
9 1.015) WNO149 VL0r4e > OL 10/9755 0:790 
10 0.760% 505760) WOl760) 057500 05745 Sr 10:613 


Table 6 presents the posterior estimates of o using the 
different priors on o;. As we can see from Table 6, when 
a, and b, are small (< = 0.01), there is almost no difference 
among the estimates at all. As a, and 5, increase, the 
estimates oO; ’ become smaller. However, if there is 
strong prior information on a, and b, for example, 
a, =6,=10, then the posterior estimates of 6, will be 
significantly different from the ones under noninformative 
priors. 


Table 6 
Comparison of Estimated Sampling Error Variances 
Small IG(a,, b,), a; = b; 
a) Oy ienOOL Reno! 1 10 
1 40.09 40.1 40.05 39.64 37.14 22.29 
2 SA ORS US S43 ee S39 asl 4, e905 
3 G4.48 9449 9442 93.76 86.73 50.60 
4 SPH} eyAYs}  « GAAS Sule Lal” | ues 
5) PASO, BD). TALON TUES ZO) GIs. 7/5) 
6 94°03) 94,0359 93'835 9 92:96") "S725 52.90) 
fl 102.30 102.30 102.20 101.40 94.85 57.58 
8 160.10 160.00 159.90 159.10 147.60 86.61 
9 63.46 63.46 63.38 62.99 5846 34.85 
10 65.88 65.87 65.89 65.40 60.76 36.60 


4. CONCLUDING REMARKS 


In this paper, we have presented hierarchical Bayes 
methods for small area estimation, using multi-level 
models. Clearly it is not easy to provide a suitable model for 
all small areas with satisfactory results, even if the Markov 
Chain Monte Carlo (MCMC) Bayesian methods such as the 
Gibbs sampling enable us to fit the data using Bayesian 
models of virtually unlimited complexity. The size and 
homogeneity of the areas and the availability of good 
auxiliary information will affect the final results. Models 
which prove suitable in some situations may be unsuitable 
in others. The hierarchical Bayes method also has some 
limitations such as the choice of priors on the model para- 
meters and some sampling issues related to the Gibbs 


sampling method. Nevertheless, the general hierarchical 
Bayes methodology is applicable to a wide variety of situa- 
tions for estimation of small area parameters. Model 
selection and choice is an important part of the hierarchical 
Bayes analysis. It is also important to compare the hierar- 
chical Bayes method with other widely used methods in 
small area estimation, such as empirical Bayes (EB) and 
empirical best linear unbiased prediction (EBLUP). Work 
is in progress on extending our work to account for survey 
design weights, along the lines of You and Rao (1999). 
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APPENDIX 


Al: 


The Rao-Blackwellized estimator of the posterior variance 
of B, is given by: 
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A2: 
Lemma: [| Y, B, t, y, 2, 4] is a log-concave function of 7. 


Proof: Let h(n) = log[n|Y, B, 1, y, Q, 2]. It is enough to 
show that 


2 
GAM go, 
on 
Clearly, 
oh(n) ry) m 
——— = -m—— + mlog(A) + log (I];.,7;). 
an rn) 
Let y(n) =I’ (m)/T'(m), then we have 
&h ; 
SOD ~ —my'(n) <0 


en 


since m>O and y’(n) is positive on (0, ©)(Temme, 
1994, 54-55). 
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Double Sampling for Ratio and Regression Estimation 
With Sub-sampling the Non-respondents 


FABIAN C. OKAFOR and HYUNSHIK LEE! 


ABSTRACT 


Cochran (1977, p. 374) proposed some ratio and regression estimators of the population mean using the Hansen and Hurwitz 
(1946) procedure of sub-sampling the non-respondents assuming that the population mean of the auxiliary character is 
known. For the case where the population mean of the auxiliary character is not known in advance, some double 
(two-phase) sampling ratio and regression estimators are presented in this article. The relative performances of the proposed 
estimators are compared with the estimator proposed by Hansen and Hurwitz (1946). 


KEY WORDS: Hansen and Hurwitz estimator; Survey cost; Optimum sampling fraction. 


1. INTRODUCTION 


In many human surveys, information is in most cases not 
obtained from all the units in the survey even after some 
call-backs. An estimate obtained from such incomplete 
data may be misleading especially when the respondents 
differ from the non-respondents because the estimate can be 
biased. Hansen and Hurwitz (1946) proposed a technique 
for adjusting for non-response to address the bias problem. 
Their idea is to take a sub-sample from the non-respondents 
to get an estimate for the subpopulation represented by the 
non-respondents. 

Cochran (1977), using Hansen and Hurwitz (1946) 
procedure, proposed the ratio and regression estimators of 
the population mean of the study variable in which informa- 
tion on the auxiliary variable is obtained from all the sample 
units, while some sample units failed to supply information 
on the study variable. In addition, the population mean of 
the auxiliary variable is known. In this paper we shall 
assume that the population mean of the auxiliary variable is 
not known. We, therefore, use the double sampling method 
to estimate the mean of the auxiliary variable and then go 
on to estimate the mean of the study variable in a similar 
manner as Cochran (1977). 

In practice, non-response is often compensated for by 
weighting adjustment (Oh and Scheuren 1983) or by impu- 
tation (Kalton and Karsprzyk 1986). The procedures used 
for weighting adjustment and imputation strive for elimina- 
tion of the bias due to non-response. However, those proce- 
dures are based on untenable assumptions on the response 
mechanism. When the assumed mechanism is wrong, then 
the resulting estimate can be seriously biased. Moreover, it 
is difficult to eliminate the bias entirely when non-response 
is confounded in the sense that the response probability is 
dependent on the survey character. Rancourt, Lee, and 
Sarmdal (1994) provided a partial correction for the 


situation. Hansen and Hurwitz’s sub-sampling approach 
does not have this defect although it costs more because of 
extra work required for sub-sampling the non-respondents. 
Nonetheless, if the bias problem is serious, the procedure is 
a viable option to address the problem without resorting to 
100 percent response, which can be very expensive. 

In the next section, double sampling ratio and regression 
estimators are considered. Generally, the double sampling 
procedure is used when it is necessary to make use of auxi- 
liary information to improve the precision of an estimate but 
the population distribution of the auxiliary information is 
not known. The first phase sample is used to estimate the 
population distribution of the auxiliary variable, while the 
second phase sample is used to obtain the required infor- 
mation on the variable of main interest. The optimum 
sampling fractions are derived for the estimators for a fixed 
cost. The performances of the proposed estimators are 
compared both theoretically and empirically with the 
Hansen and Hurwitz estimator. 


2. THE DOUBLE SAMPLING RATIO AND 
REGRESSION ESTIMATORS 


2.1 Background 


To estimate the population mean Y of the auxiliary 
variable, a large first phase sample of size n’ is selected 
from N units in the population by simple random sampling 
without replacement (SRSWOR). A smaller second phase 
sample of size n is selected from n’ by SRSWOR and the 
character y is measured on it. The ratio estimator of the 
mean of y is y’, = (y/x) x’, where x’ is the sample mean 
from n’ units, y and X are obtained from the second phase 
sample if there is no non-response in the second phase 
sample. If, however, there is non-response in the second 
phase sample, we may use an estimator obtained from only 
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the respondents or take a sub-sample of the non-respon- 
dents and re-contact them. The former option is much 
cheaper than the latter because securing missing informa- 
tion from the non-respondents by re-contact requires 
usually much more effort and cost. However, it is quite 
feasible that the non-respondents differ significantly in the 
main character from the respondents so that a serious bias 
results. In this situation, sub-sampling of the non-respon- 
dents may be beneficial. Hence, we pursue the sub-sam- 
pling idea of Hansen and Hurwitz for a double sampling 
situation. Basically, the estimators proposed here are double 
sampling version of Cochran (1977, p. 374), that is, double 
sampling ratio and regression estimators for Y adjusted for 
non-response by using the Hansen and Hurwitz (1946) 
procedure. 

Let’s assume that all the n’ units supplied information 
on the auxiliary variable x at the first phase. But let 7, units 
supply information on y and n, refuse to respond at the 
second phase. From the n, non-respondents, an SRSWOR 
of m units is selected with the inverse sampling rate k, 
where m =n,/k,k> 1. All the m units respond this time 
around. This can be applied in a household survey where 
the household size is used as an auxiliary variable for the 
estimation of, say, family expenditure. Information can be 
obtained completely on the family size during the house- 
hold listing while there may be non-response on the house- 
hold expenditure. 

In the following presentation, we assume that the whole 
population (denoted by A) is stratified into two strata: one 
is the stratum (denoted by 4,) of N, units, which would 
respond on the first call at the second phase and the other 
stratum (denoted by A, ) consists of N, units, which would 
not respond on the first call at the second phase but will 
respond on the second call. Let the first and second phase 
samples be denoted by a’ and a respectively, and let 
a, =af\A, and a, = a{\A,. The sub-sample of a, will be 
denoted by a,,. Summation over the units in a set s will be 
denoted by 3 

As a general rule, population parameters are denoted by 
capital letters except for Greek letters and the sample 
statistics by corresponding small letters. 


. 


2.2 The Double Sampling Ratio Estimator 


We define the double sampling ratio estimator as 
follows: 


* 


qd’ =2 


x 


(2.1) 


x’ = r*x’ 


where X* and y* are the Hansen-Hurwitz estimators for X 
and Y, respectively, and are given by 


Hew? , 2 2.2 
Uu =W,U, + WU, uU=X,y. ( ) 


According to the general rule, we define W, = N./N and 


: ae vil 
Was nj! n, j = 1 or 2. Sample statistics obtained from a 


are subscripted by “2m”, (e.g., ,,, = (1/m) Mii those 
from a, are subscripted by “1”, (e.g., u, = (lin) d,, u;), 
and those for the first phase sample a’ will be super- 
scripted by a prime (e.g., ¥' = (1/n')X_.x,). 

A large sample first order approximation to the variance 
of d*, obtained by using the Taylor linearization, is given 


by 


ae al Up eee) ee ee 
Vd") =| —-— |S, +] -—-—|S 
a) nia 


n n 
W,(k-1 
eA hae) s2 (2.3) 
n 
where, 
S, =S, + R75, 2RS0) 
(2.4) 


Sy, = Sy + R7S;, ~ 2RS;,_, 
R is the population ratio of Y¥ to X. S? and S,, are, 
respectively, the variance for the whole population and the 
population variance for the stratum of non-respondents of 
the variable w. oY and S,_ are the covariances for the 
whole population and the population of non-respondents 
respectively. 

The variance of d* can be approximately estimated by 


A xa a Wa alee 
vay-{+-1)s.(4-1)¢.7 De 
n 


r 
Tt hn n 


(2.5) 


s= —y, y th ¥) -V? + wok ey! 


n-1 
$= : >» (y,-r*x, +k>, (y,-r°x,)'}and 
FI a I 1 Fyn, Ul Ul 
cspaliaind Bou ter (2.6) 
m-\ 2m 


Note that sy is an unbiased estimator of s, . It seems 
natural to use Se to estimate We since the expression 
obtained from S*, by replacing r* with R is a consistent 
estimator of S,. The same argument can be used to justify 
the use of S ai 

An alternative estimator of V(d*) can be obtained by 
replacing S and oe with 
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respectively, in (2.5), where, 


’ 


1 am 
58 ee 0; Toke 2: 
2 
1 reeds: 
ora Oe XY; am Fo 


and SS, is as in (2.9). This alternative estimator is likely to 
have a smaller variance than the estimator in (2.5) since the 
estimators s/? and cs are based on larger samples and 
therefore more precise. 


2.3 The Double Sampling Regression Estimator 


We define the regression estimator by 
(2.8) 
where B is an estimator of B = S_/S,. > There could be 


several choices for BY, but a natural choice would be given 
by 8 = = Sy, Is" , where 


5, = = yee xy, a, Hy, nay" Jand 


(2.9) 


Caeae| 2 2 = 
ae ( a ae thy x; — Nxx 


Iti is easy to show that Se and s, * are unbiased for S_, and 
Se respectively. An approximate variance of ¢” is given as 


tala (<a al Wl ti 
PSEA PRES IGE see [hes Rad I 


_ ED) s2 (2.10) 


where Se and Se are obtained from (2.4) by replacing R 
with B. 
To estimate V(t") we can use the following formula: 


1 1 | 32 1 1 \ s2 
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(2.11) 
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where, 


1D, ,-¥) +kD, O-H) y, 


ie *) and 


eee xh), (2.12) 


Like (2.7), a slightly improved estimator of V(t*) can be 


obtained by using 


Fe 2 Ax2 2 Ax 
Sy Sia Poe Somxy ey) 


3. CHOICE OF SAMPLING FRACTIONS 


We shall now deduce the optimum &, n, and n’ that 
minimize the variances of the proposed estimators for a 
specified cost, or that minimize the cost for a specified 
variance. 

Let’s consider a cost function for d* given by 

C=cn 7h cn tom 


(3.1) 
where the c’s are the costs per unit defined as follows: 


c’ : the unit cost associated with the first phase sample, 
a 

c : the unit cost of the first attempt on y with the second 
phase sample, a; 


c, : the unit cost for processing the respondent data on y 
at the first attempt in a, ; 
c, : the unit cost associated with the sub-sample, a, of 


a. 
Since the value of , is not known until the first attempt is 
made, the expected cost will be used in the minimization. 
The expected cost is given by 
c,W, 
n 
k 


The optimum values of k, n, and n’ that minimize the 
variance of d* for a fixed expected cost C* are obtained by 
using Langrange multiplier. The optimum values thus 
obtained are: 


(3.2) 


Be)=c'sen' afore 
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2 2 
os ra W,S;,) 
Sy.(c +¢,W,) 


ne= cA and n, = 
0 DYG (353) 
where 
A=S? +W,(k, - 1)S3, 


c,W, 
Gero + r and 


oO 


D-= y(S, -S’)c' + JAG. 


Ifwelet y = c,/(c +c, W,),8 =S, /S>, and & = S//S,, then 
we have 


Lge fr(5 = W. 


C*,/1 + Wi(k,- 1)/5 
n iain pe Le toa Se ay pone op 


VGe'"(E-1) +GJ1T+W,, - D/8 
ane Cyé-1 | 
*  e'E-1 + JGe"(1 + WK, - 178) 


The optimum values n, and 7; are proportional to the 
expected cost, C*. To get the optimum values of k,n, and n’ 
that, minimize V(t") we simply substitute Se and Se in 
the above expression in (3.3) with S A and S,,, respectively. 
Table 1 shows optimum values of k,,n,, and n/ for given 
parameters. 


(3.4) 


Table 1 
Optimum Values of k,,n,, and n} 
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4. COMPARISON OF THE ESTIMATORS 


In this section, the theoretical comparison of the perfor- 
mances of the proposed estimators with respect to the 
Hansen and Hurwitz (1946) estimator is made first without 
taking the cost into consideration and then with taking it 
into account. 


4.1 Without Considering the Cost 


The variance of the Hansen-Hurwitz estimator is 


even algae W(k - 1) 
Vy") -( as ay (4.1) 


where jy" is defined as in (2.2). 
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This is positive (i.e., d* is more efficient than jy”) if 
R<2B and R<2B,, where B, = Soy So On the other 
hand, we have 


aS 
vo)-vey-(4-4) xy 


+ Wee Sh (2B, - B). (4.3) 
Therefore, ¢* is more efficient than the Hansen-Hurwitz 
estimator if (4.3) is positive. One particular condition under 
which this can occur is that B, > B/2 with B20. The 
conditions we discuss here are sufficient and thus, d* or ¢* 
can be more efficient than y* under more relaxed 
conditions. 


4.2 Considering the Cost 


We shall now compare the proposed estimators with the 
Hansen-Hurwitz estimator (y*) making use of the cost 
function given in section 3. 

For the estimator 7", if a straight random sample is taken 
(without using double sampling procedure) for y, the 
optimum sample size for an expected cost, 


Cc, W. 
cel eran: sey 


similar to the one in (3.2) can be obtained by the same 
technique (i.e., Langrange multiplier) used in section 3 as 
follows: 
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Then the optimum variance of the Hansen-Hurwitz 
estimator becomes 
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If we compare this with V(d*) with the optimum choices of 
k,n, and n’ in (3.3), then the condition that d* will be 
more precise than y” is given by 
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and p is the correlation coefficient between x and y. 
We can obtain a similar comparison between jy" and ¢”. 
That is, ¢* is more efficient than y* if 


x 


2p -Bh> 


1-6, 


me a 0, a Q, is 0, Qhny)~ hQ2 B, rs ph (4.7) 


4.3 Empirical Comparison of the Proposed 
Estimators 


The relative efficiencies of the estimators d* and f* with 
respect to y* are compared using an artificially generated 
population. The parameters of the population are: 
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R = 1.92, B = 1.52, p = 0.85, R, = 1.88, B, = 1.47, 
p, = 0.83, N = 1,000, N, = 302, S? = 766.54, 
S, = 2426.82, S,, = 1164.08, S;, = 433.63, 


S3, = 1350.05, and S,,, = 638.32. 


The relative efficiencies of d* and ¢* are presented in 
Table 2. Note that R is substantially different from B, which 
means that the regression line does not pass through the 
origin. Under this population, the regression estimator f* 
is more efficient than the ratio estimator d*. We notice also 
that the optimum initial sample size, n/ for t* is more than 
for the estimator d*. The reverse is the case for the 
optimum second phase sample size n,. This is so because 
the regression estimator can be more precise with a smaller 
second phase sample size so that it allows to allocate more 
to the first phase sample. Finally the optimum inverse 
sampling rate k, is practically the same for the two 
estimators. 

When the linear regression line passes through the 
origin, the advantage of t* over d* disappears, as expected 
and confirmed in another empirical comparison not shown 
here. 


Table 2 
The Relative Efficiencies of d* and ¢* with 
Respect to y* (C* =200, c=0.5, c, =1) 


GC MLS US n, n, Efficiency 
7D 

0.1 a Ses SAT ESS 92 514 1.85 

0.1 4 1p). Mil PAS 85 477 1.91 

0.3 2 1.58 127 1.46 78 250 23S) 

0.3 4 223 Wise 2.06 18 234 e357 
i" 

OF 2 58s) 27 1.47 89 563 2.11 

0.1 4 223 PS 2:08 83 523 2.19 

0.3 2 1S Sel 7), 74 269 1.34 

0.3 4 DS el See 0S 70 253 1.45 


5. CONCLUSIONS 


We proposed ratio and regression estimators based on 
the double sampling procedure when there is non-response 
on the main character and the population mean of the auxi- 
liary variable is not known. The potentially serious non- 
response bias is eliminated by sub-sampling the non- 
respondents as in the Hansen and Hurwitz procedure 
(1946). We derived optimum sample sizes for a given set of 
unit costs and compared theoretically and empirically the 
performance of our estimators with that of the Hansen and 
Hurwitz estimator. 
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When there is a strong linear relationship between the 
main character and the auxiliary character and the auxiliary 
data can be collected cheaply with a large sample size, our 
estimators are substantially superior to the Hansen and 
Hurwitz estimator. Our procedure can be useful when there 
is a serious concern about the nonresponse bias that is 
difficult to handle with the usual weighting adjustment or 
imputation. 
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Modeling Interviewer Effects in Panel Surveys: 
An Application 


JAN PICKERY and GEERT LOOSVELDT' 


ABSTRACT 


In this paper we will combine two applications of multilevel models. The multilevel model is suitable to analyze interviewer 
effects on survey data. It can also be used to analyze longitudinal — “repeated measurements” — data. We will analyze a data 
quality indicator of panel data that come from the Belgian Election Studies. These panel data consist of only two waves. 
The respondents that cooperated twice are for the most part not interviewed by the same interviewers. This results in a 
complex data structure with measurements nested in respondents, and respondents nested in interviewers, but without an 
overall hierarchical nesting structure: cross-classification. This complicated data structure will be analyzed in two different 
ways: an analysis of all respondents and an analysis of only those who are interviewed twice by the same interviewer. The 
results of these different analyses will be compared. We conclude that the multilevel cross-classified model is a very flexible 


and useful tool to analyze interviewer effects in panel surveys. 


KEY WORDS: Multilevel models; Cross-classifications; Panel surveys; Interviewer effects; Don’t know answer. 


1. INTRODUCTION 


In this paper we analyze the effect of respondent and 
interviewer characteristics on the number of “don’t know” 
answers in two waves of the panel survey from the Belgian 
Election Studies. We use different multilevel models for a 
subset of the dataset and for the entire dataset. The main 
purpose of the article is to illustrate how interviewer effects 
in a panel survey can be analyzed using multilevel models. 

A multilevel or hierarchical model is an appropriate tool 
to analyze data with nested structures, e.g., pupils nested in 
schools or patients in hospitals. A multilevel model can 
include variables of the different levels of nesting, but it 
also takes account of the variability associated with each 
level. The typical quality of the models is not the functional 
form relating the variables of the different levels, but rather 
a more sophisticated treatment of the error structure 
(DiPrete and Forristal 1994, 334). In education research for 
instance a multilevel model can account for variation 
between schools and variation between pupils. Moreover 
the model tries to replace this variance attributed to both 
levels by variables of either level. These models are des- 
cribed in various textbooks like Bryk and Raudenbush 
(1992), Goldstein (1995), Kreft and de Leeuw (1998) and 
Snijders and Bosker (1999). 

Multilevel or hierarchical models also offer the best 
possibilities to analyze interviewer effects on survey data 
(Hox 1994). A hierarchical model is the best tool to tackle 
the “respondents nested within interviewers” — design. 
Other statistical techniques require mutual independence of 
interviewer and respondent characteristics, which is — most 
of the time — not the case because of the hierarchical struc- 
ture of the data. In a multilevel model both the regression 


coefficients and the variance components are conditional on 
the explanatory variables in the model, which is a useful 
property if there is no complete orthogonalization of inter- 
viewer and respondent variables (Hox 1994, 307). When 
respondents are not randomly assigned to interviewers, 
respondent and interviewer characteristics can become con- 
founded since respondents from a specific area will most 
likely be interviewed by interviewers from the same area. In 
such a situation, if the relevant respondent variables are put 
in the multilevel model, interviewers are equalized by statis- 
tical means. For that reason the assumptions of an analysis 
of interviewer effects with a multilevel model are more 
realistic than those of an ANOVA or ANCOVA. Further- 
more the hierarchical model allows estimation of both the 
interviewer variance and the effects of explanatory vari- 
ables measured at the interviewer and the respondent 
model. This possibility of replacing variance attributed to 
respondents/interviewers with the effects of respondent/ 
interviewer characteristics allows for wider generalizations. 

The multilevel model can also fruitfully be used to 
analyze longitudinal — “repeated measurements” — data (see 
e.g., Goldstein 1995, 87-95; Snijders 1996 and Yang and 
Goldstein 1996). There are alternatives to analyze the 
“measurements nested in individuals” — design, but multi- 
level analysis has some clear advantages. Using a hier- 
archical model, it is feasible to handle unbalanced designs 
— not all individuals have the same number of measure- 
ments — and quite easy to incorporate changing covariates. 
Besides, the model allows more nesting levels. The 
individuals can be nested in another higher level unit. 

We will analyze respondent and interviewer effects on the 
number of “don’t know” answers on a series of questions 
regarding political parties in a panel survey. We have 
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measurements (wave | and wave 2) nested in respondents 
(the longitudinal design) and respondents nested in inter- 
viewers. Our panel data consist of two waves. During the 
second wave the respondents are for the most part not inter- 
viewed by the same interviewers. The purely hierarchical 
nesting has broken down and a more complex data structure 
is the result. To handle this data structure it is necessary to 
conceive the measurements as being nested into two 
different classification structures: measurements nested in 
respondents and in interviewers. This is called a cross- 
classified design, because the nesting of the levels is not 
purely hierarchical. 

In this article we'll start with a simple data structure and 
the appropriate model. Afterwards the model will become 
more complex. We’ll perform two analyses. In our first 
analysis we work only with the respondents who are inter- 
viewed twice by the same interviewers. Afterwards we will 
analyze all the respondents, including those who were inter- 
viewed only once. In the first analysis the purely hier- 
archical structure remains intact. The model is a “simple” 
three level one: measurements nested in respondents nested 
in interviewers. Analysis 2 sets up a cross-classified model. 
In that model the measurements are classified by respon- 
dents and interviewers. 

The next section reflects on the nature of our dependent 
variable, the “don’t know” answer, and the way to analyze 
it. In the third section we’ll describe our data in detail to 
clarify the complex structure. The following section (4) 
treats in brief the different models that we will combine. 
Section 5 presents the variables in our analysis. In sections 
6 and 7 we discuss the setup of our 2 different models and 
report the results of the analyses. Section 8 concludes the 
article. 


2. ANSWERING “DON’T KNOW” 


It has become generally acknowledged that the use of a 
“don’t know” or a “no opinion” filter increases the propor- 
tion of respondents who give this answer, and that the 
increase itself is a function of the nature of the filter used 
(Schuman and Presser 1981, 143). Krosnick argues that 
answering “don’t know” is one form of satisficing. Satis- 
ficing occurs when a respondent is not motivated to expend 
the mental effort necessary to generate optimal answers. A 
“no opinion” answer is an acceptable answer but it is the 
result of a “weak” cognitive process. Satisficing is a 
function of task difficulty, and the respondent’s lack of 
knowledge, ability and motivation. This theoretical 
reasoning is consistent with the finding that offering a 
“don’t know” response option increases the proportion of 
respondents who select it, particularly among respondents 
with little formal education and people who consider an 
issue to be less personally important. (Krosnick 1991). 
Following this argumentation, answering “don’t know” is 
mainly explained by respondent characteristics that can be 


related to the cognitive aspect of answering questions. 
Previous research points us to the following characteristics 
of interest: education (e.g., Sudman and Bradburn 1974), 
age (see e.g., Groves 1989, 441-443), sex (e.g., Hox, 
de Leeuw and Kreft 1991), and a measure of involvement 
or interest in the subject (e.g., Groves 1989, 419). 

However answering questions is not only a cognitive 
process of the respondent but it is also a communicative 
process (Schwarz and Sudman 1995). Within this process 
the interviewer plays an important role. There is a lot of 
literature about the interviewer as a source of survey 
measurement error (Groves 1989). The main idea is that 
interviewers are not “neutral” collectors of data but that 
they can influence the respondents’ answers. Item non- 
response too is subject to interviewer effects as has been 
shown long ago by e.g., Hanson and Marks (1958) and 
Bailar, Bailey and Stevens (1977). A social scientist 
interested in the explanation of “don’t know” answers 
should therefore include respondents and interviewers in 
the analysis. The number of “don’t know” answers will be 
the dependent variable of our analyses. 


3. DESCRIPTION OF THE DATA STRUCTURE 


After the 1991 General Election in Belgium a national 
survey was set up in which 4,544 face to face interviews 
were conducted in the three Regions in the early months of 
1992. A two-stage self-weighting sample (see e.g., Sarndal, 
Swensson and Wretman 1992, 141-144) was used. The 
sample was representative for the population of 18-74 years 
old (ISPO/PIOP 1995). In this article we will use the data 
from the Flemish region, which cover 2,691 Flemish 
respondents, interviewed by 163 interviewers (Carton, 
Swyngedouw, Billiet and Beerten 1993). After the 1995 
Elections a similar survey was set up. Due to budgetary 
constraints the sample had to be smaller for the second 
wave. So the 2,691 respondents were used as a group to 
sample from and, in second order, there had to be new 
respondents to compensate for the aging of the youngest 
cohort from 1991. Finally 2,099 respondents were inter- 
viewed by 167 interviewers. This sample contained 1,762 
panel respondents and 337 new respondents (see Beerten, 
Billiet, Carton and Swyngedouw 1997 for a detailed 
technical report of the sample plan). Only 55 of the inter- 
viewers of the first wave collaborated again. So there were 
112 new interviewers in the second wave. 

This gives us a dataset with 3028 respondents (2,691 + 
337) and 275 interviewers (163 + 112). For 1,762 respon- 
dents we have a measurement in both waves, for the rest 
(1266) there is only one measurement. The structure of the 
dataset can be represented in a table similar to table 1 (see 
also Goldstein 1995, 114). Each x in the table reflects an 
observation. The complete dataset contains 4,790 observa- 
tions ((1,762 x 2) + 1,266). Each type of respondent in the 
table represents a possible occurrence in the dataset. 
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Table 1 
A Representation of the Dataset 
Respondent Respondent Respondent Respondent 
Type 1 Type 2 Type 3 Type 4 
N 


(Inter- 
Wave 1 2 1 2 1 2 viewers) 
Interviewer 
Type A 
Interviewer 
Type B 
Interviewer 
Type C 
Interviewer 
Type D 


N (Respondents) 374 1388 929 337) 


47 


This table illustrates that we have three kinds of respon- 
dents: panel respondents who are interviewed twice by the 
same interviewer (Type 1), respondents who cooperated 
twice but were interviewed by different interviewers 
(Type 2) and respondents who are only interviewed once 
(Type 3 and 4). Our 2 different analyses are based on these 
different types of respondents. In Analysis 1 we’ll look at 
the respondents whose situation corresponds with that of 
Type 1. Only 374 Respondents satisfy this condition. They 
were interviewed twice by the same interviewer. Analysis 
2 takes all the 3028 respondents into account (Respondents 
1 to 4 of the table). 

Furthermore the table shows that we can also discern 
different interviewers: interviewers who collaborated twice 
(Type A and B) and interviewers who collaborated only the 
first wave (Type C) or only the second wave (Type D). The 
interviewers of Type B collaborated in both waves, but 
never interviewed the same respondents twice (unlike the 
interviewers of Type A). 

To analyze this complex data structure we will combine 
three different models that are presented in the next section. 


4. ASHORT DESCRIPTION OF THE 
DIFFERENT MULTILEVEL MODELS 
USED IN THE ANALYSES 
4.1 The General Multilevel Model 


The first model we need is the general multilevel model, 
which has the following form: 


tae PoP yet Sy (1) 


Bo; =Bo + M%, and B,,=B, +4, (2) 

or 
Bo; =Bo + Yor 21 * Mo, and By =Bit+%2, +4). (3) 
Subscript i refers to the level 1 unit and subscript j to the 


level unit 2. In our situation level 1 indicates the respondent 
and level 2 the interviewer. So the response variable Y of 
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respondent i, interviewed by interviewer j is dependent on 
the x variable of that respondent. This relationship looks 
like an ordinary regression model but the parameters of the 
model are interviewer specific. The B’s differ from inter- 
viewer to interviewer. For each f, there is an interviewer 
residual (1), or u,,). The B’s can also be made dependent 
on higher level variables (interviewer characteristics), 
allowing for generalization across interviewers. We have 
one second level variable z. Substituting (3) into (1) results 
in the following overall model: 


= + + 
Y= Bo * Bixy * Yor2y * Yury ry + My X1y * Yay + &y- (4) 


Of course more x and z variables can be included in these 
relationships. We assume that the residuals Upj> Uy, and e,, 
have means 0 given the values of the explanatory variables 
z and x. Furthermore it is assumed that the level 1 residuals (é,,) 
are independent. The level 2 residuals (Up oy and uy, .) are 
assumed to be independent from ei and to have a joint 
multivariate normal distribution with covariance matrix 2. 
They don’t have to be independent from each other. Usually 
they are correlated. 


4.2 The Multilevel Model for Longitudinal Analysis 


The second model we need is the longitudinal multilevel 
model. In an analysis of a “repeated measurements” 
design with a hierarchical model, the measurements are 
considered to be the first level and the individual the 
second. Most of the time the individual units will be 
persons, but of course they can be other units, like e.g., 
schools or countries. In our analysis the individuals are the 
respondents. The analysis tries to estimate a growth curve 
on the base of the different measurements and to compare 
differences in curves given individual characteristics. Each 
observed value is made conditional upon the time of mea- 
surement — which can be a measure of time, but also age — 
and possible transformations of this measurement. Usually 
the curve is assumed to be a polynomial, which has the 
following form: 

Y,, = To, + Hf + Ty t? iyeet Teh + Cis (5) 

Y,, is the observed value for respondent i on moment f, 
tcan ‘be time of measurement or age. 7,,(h = 0...k) are the 
trajectory parameters or growth parameters for subject i, k 
is the degree of the polynomial. In a simple case k has the 
value 1 and then there is a linear curve. If there are m 
moments of measurement, a polynomial with degree of 
m — 1 will result in an exact reproduction of the curve. Of 
course it is more interesting to use a polynomial with a 
lower degree if that yields a satisfactory reproduction of the 
curve. You can test whether the model with degree k + 1 
results in a significant improvement compared to the model 
with degree k. 

The growth parameters have aiso a subscript for the 
individual (respondent). The model states that these para- 
meters differ from individual to individual. The second part 
of the model defines these parameters: 
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‘akin ee ak (6) 
or 
Ty; = TM + Boi X41) * oi: (7) 
The individual parameter equals a general parameter 
(x,)+ an individual residual (7),). By the inclusion of 
individual characteristics (x) it may be possible to reduce the 
individual specific part, thus generalizing across respon- 
dents. In line with (1) and (2) we chose x to denote the indi- 
vidual (respondent) characteristics. But it is worth mention- 
ing that in this model the x variables are higher level (level 2) 
variables. The individual characteristics can be fixed (the 
same for all moments of measurement) or varying. 


4.3 Cross-Classified Models 


The third model we will use is the cross-classified 
model. Not all data structures are purely hierarchical. Units 
may be classified along more than one dimension (see 
Goldstein 1995, 113-116). For example students can be 
classified by the school they. go to and by the neighborhood 
they live in. In our example measurements are classified by 
respondents and by interviewers. A cross-classified model 
has the following form (subscripts 7, and 7, refer to the 2 
different classification structures): 


Fi, = Porgy * Py, “uh, + Suis? (8) 


Boj, = Bo + Moy, + Moy, 2d By) = By), * My, * y,- (9) 


Equation (9) can be reformulated the same way as 

equation (3). 
Y;;,;, 18 the observed value for individual 7, classified by /, 
and j,. In our case: the observed value for measurement i 
on respondent /,, interviewed by interviewer j,. The para- 
meters associated with the independent variable x have a 
residual for both classifying structures. For this model the 
additional assumption is made that the residuals of the 
different classifying structures (in our case: the respondent 
and interviewer residuals) are mutually independent (up, 
and uy, versus ug, and u,, ). 

Raudenbush (1993) discusses this kind of models and the 
use of the EM algorithm to estimate them. Rasbash and 
Goldstein (1994) and Goldstein (1995, 123-124) show how 
these models can be specified and estimated using a purely 
hierarchical formulation and (consequently) standard multi- 
level software. The way to do this is to specify one of the 
classifications as a standard hierarchical one, then define a 
dummy for each unit of the other classification, specify that 
each of these dummy variables has a random coefficient at 
the higher level and constrain the resulting sets of variances 
to be equal. 

In section 6 and 7 we’ll use these 3 different models. 
They can all be implemented in MLn/MLwiN, software for 


multilevel modeling. Firstly we take a closer look at the 
variables we will use in the analysis. 


5. VARIABLES IN OUR ANALYSIS 


One of the more difficult tasks during the interview of 
the election study was rating six parties on different 
11-point scales. Three scales were presented to the 
respondents: catholicism, economic liberalism and feder- 
alism. An explicit “don’t know” filter was included in the 
question, but it was not mentioned on the card with the 
alternatives given to the respondent. The entire question is 
included in the Appendix. We expected a considerable 
number of “don’t know” answers because of the degree of 
complexity of the task. The explicit filter was expected to 
raise that number as well (see e.g., Schuman and Presser 
1981). 

In the first wave the average number turned out to run up 
to more than 4 “don’t know” answers per respondent. 
Almost 20% of the respondents made use of this possibility 
at least 9 times out of the 18. If we consider only the panel 
respondents the mean number is a bit lower (3.8). This is 
not surprising since we could expect that “multi-users” of 
the “don’t know” answer would be underrepresented in the 
second wave because of lack of interest in the subject of the 
survey and/or difficulties in answering the questions. In the 
second wave the overall mean is 3.6 and the mean for the 
panel respondents 3.4. The numbers for the respondents 
that were interviewed twice by the same interviewer are 3.9 
and 4.2 respectively. ‘There is no explanation why the 
number of “don’t know” answers during the second wave 
is higher than the overall mean for these respondents. 

At the measurement level we’|l use the year of the inter- 
view as indication of time of measurement. We’ ve recoded 
this variable, so time has the value 0 for the first wave and 
3 for the second wave. 

At the respondent level we have 3 independent variables: 
sex (0 = man, 1 = woman), completed education (0 = low, 
1 = high) and the extent to which the respondents follow 
political news in the press (press: 1 = (almost) always - 5 
never). The first 2 variables are constant for the 2 times of 
measurement. The third is a time-varying covariate and the 
question phrasing also slightly changed for the second 
survey. The two different questions are also included in the 
appendix. The dissimilarity in the phrasing induces an 
additional difficulty in setting up the model. The way to 
handle such a variable is to standardize it (mean 0, variance 
1) for each time of measurement and (afterwards) to ascribe 
the value 0 to the time of measurement when the question 
wasn’t asked. The reference value for those variables is 
their mean (see Snijders 1996, 422). This gives us 2 
variables: press! for the first occasion and press2 for the 
second. The former has the value 0 for all respondents for 
the second measurement and the latter for the first measure- 
ment. We don’t take up the respondent’s age in the model, 
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since this variable would correlate too much with the time 
measurement at the occasion level. 

In order not to complicate the analysis too much we 
don’t take up interviewer variables. We just assume there is 
an interviewer effect, without trying to explain that effect in 
terms of interviewer characteristics. 


6. ANALYSIS 1: RESPONDENT TYPE 1 OF 
TABLE 1 


The first analysis considers only those respondents who 
are interviewed twice by the same interviewer (cfr. Type 1 
from Table 1). This analysis requires a “simple” three level 
model: measurements nested in respondents nested in inter- 
viewers. The hierarchical structure is unambiguous. This 
model is similar to the example in chapter 8 in the Bryk and 
Raudenbush book (1992). In that example the authors 
analyze the progress in academic achievement of students 
in schools. 

Our dependent variable is the number of “don’t knows’’s 
for respondent i on moment ¢, interviewed by j (Y,, ip): We 
have only 2 measurements so the degree of the polynomial 
cannot exceed 1. This results in the following level 1 
equation: 


it 


ti = Moy * Mi YEAR + en: 


Our time variable (f) is the year of the interview which 
has the value 0 (1992) or 3 (1995). We will test whether 
My iS peouicany, If not, this leaves us a null model or 

“naive” model (see Snijders 1996, 411), in which the 
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number of “don’t know” answers doesn’t change over time, 
and both measurements can be considered as retests of the 
same constant value. The coefficients in the level 1 
equation are respondent and interviewer specific. 

At the respondent level we’ll include 3 variables: sex, 
education and the 2 press variables. So our level 2 equation 
contains 4 variables: 


Toy = Mo + Boy; SEX, + Bo), EDUCATION, 
+ Boz, PRESS1, + Bog; PRESS2; + 79,.. 


If the parameter estimate associated with year is signifi- 
cant we’ |l have a similar equation for To 

At the third level (interviewer) we won’t include any- 
more variables, but we will fit a random intercept and 
random slopes. So we have the following level 3 equations: 


Mo = My + Up, aNd Boy; = Boy + Moy) --- 


Implementing these model specifications in MLn gives 
us the following results. 

Model a in the table is the null model. This is a model 
without independent variables, neither at the measurement 
level, nor at the respondent level. In this model there is no 
evolution in the number of “don’t know” answers. But the 
variance of the dependent variable is divided in a measure- 
ment part, a respondent part and an interviewer part. All 
variances are significant. This indicates that there is 
between wave variation, that some respondents use the 
answer more than others and that some interviewers will get 
more “don’t know” answers than other interviewers. 


model c 


3.749 (0.442) 


model d 


3.754 (0.523) 


Table 2 
Analysis of Respondents who were Interviewed Twice by the Same Interviewers (s.e. in brackets) 
model a model b 
Fixed 
Measurement level 
constant 4.136 (0.322) 4.028 (0.358) 
year 0.072 (0.089) 


Respondent level 


sex 
education 
press] 
press2 

Random 


Interviewer level 


?) 


Oconstant 


Ogaucation/constant 


6, 


education 


Respondent level 


2 
Sconstant 


Measurement level 


Z 


2.249 (1.040) 


14.470 (1.714) 


o, 13.320 (0.974) 
5) Oe 4519.35 
A df’ 


Note: * compared to model a 


2.251 (1.043) 


14.480 (1.714) 


13.300 (0.974) 
4518.62 


2.393 (0.434) 
-1.675 (0.425) 
0.911 (0.263) 
1.483 (0.236) 


2.666 (0.969) 


8.939 (1.308) 


13.270 (0.969) 
4414.52 
4 


2.458 (0.414) 
-1.778 (0.446) 
0.887 (0.233) 
1.426 (0.234) 


6.090 (2.109) 
-4.099 (1.816) 
1.396 (1.819) 


8.692 (1.332) 


13.250 (0.969) 


4395.68 
6 
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The inclusion of the variable YEAR does not provide a 
better fit of the model. The decrease in deviance (-2 Log L) 
is not significant, neither is the parameter of the variable 
significant (model b). We can conclude that there is no 
significant overall evolution in the number of “don’t 
know’’s. We can go on with a model without the time 
variable. 

The respondent variables do result in a considerable 
improvement of fit of the model. The decrease of the - 2 
Log L value is large and clearly significant (p<0.001). 
According to the analysis women use the “don’t know” 
alternative more than men do and highly educated respon- 
dents less than respondents with lower education. Fol- 
lowing the political news in the press reduces your chance 
to answer “don’t know”. Both press1 and press2 are signifi- 
cant (model c). The inclusion of respondent variables also 
results in a substantial decrease of the variance at the 
respondent level. 

We also tried to fit random slopes at the interviewer level 
(model d). Our analysis showed some variation in the para- 
meter associated with the respondent’s education. This is 
the only independent variable with a varying coefficient at 
the third level. Cue ee is not significant, but there is an 
important covariance between the residual for the constant 
and the residual for education (6,5, cation/constant = ~4:099): 
The covariance is negative, indicating that interviewers 
with a higher constant have a smaller coefficient for educa- 
tion. Since the fixed parameter for education is negative, it 
will be even more negative for those interviewers, thus 
having a larger absolute value. Hence for interviewers who 
stimulate more “don’t know” answers, the difference 
between less educated respondents and more educated 
respondents will be larger. In model d the value of G7 on stant 
at the interviewer level has increased considerably, com- 
pared to model c. In this model the variance at the inter- 
viewer level is dependent on the values of the explanatory 
variable education and it will be larger for zero values of 
education. That is another interpretation of model d: the 
variance between interviewers is much higher for lower 
educated respondents than for higher educated respondents. 
This model with a more complex variance structure at level 
3 has a better fit than the previous models. 

When including YEAR in model c or model d, it turned 
out to be not significant either. Also in our final models 
there is no evidence for an evolution in the number of 
“don’t know’’s between the two waves. All models prove a 
significant interviewer effect. But the relative size of the 
variance shows that there is more variation between 
respondents than between interviewers. 


7. ANALYSIS 2: ALL RESPONDENTS 


In this analysis we look at all the respondents: the panel 
respondents that were interviewed twice by the same inter- 
viewer, the other panel respondents and those who were 


interviewed only once. This second analysis breaks down 
the hierarchical structure. Measurements are still nested in 
respondents and respondents are still nested in interviewers. 
But there is no overall hierarchical structure, since the 
interviewer can (and most of the time will) change between 
the two waves (see section 3). Our dependent variable is 
still the number of “don’t know’’s of respondent i inter- 
viewed by j on moment ¢ (Y, ij)* But the model has changed. 
The level 1 equation hasn’t: 


Yi = To, + Ty; YEAR + en: 


In this notation we use 7, since the level 1 model is also 
a growth curve. But this equation matches the level 1 model 
of the cross-classified model (equation (8), section 4.3). 
Furthermore we still use i for the respondent and j for the 
interviewer. But it is important to notice that this is not the 
same model as the one of analysis 1. These subscripts 
correspond to the j, and j, of equations (8) and (9). 

There is no “real” third level. To fit the cross-classified 
model in MLn, we have to define a third level, but concep- 
tually the respondent and interviewer are at the same level 
in this model. This leads to the following level 2 equation: 


Toy = Ho * Bo, SEX; + By) EDUCATION, 
+ Bo; PRESS1; + By, PRESS2; +79, +79). 


The interviewer specific part (7) is included in the 
second level, so there is no interaction between the inter- 
viewer variance and the respondent variables. This is the 
main difference with analysis 1. 

A cross-classified model requires enormous computa- 
tions. We have 3,026 respondents and 275 interviewers. 
This would mean 275 dummies with all varying coefficients 
at the artificial third level. Up till now it is impossible to fit 
such a model. The storage required by the worksheet is far 
too large (see Goldstein 1995, 118 and Rasbash and 
Woodhouse 1996, 85-86 for details). It is possible to reduce 
these storage requirements and improve the speed of model 
estimation by dividing the dataset in subsets in which the 
cross-classification implies fewer cells. In our case we look 
for separate groups of measurements that are classified by 
fewer respondents and interviewers. The analysis of 1 
group of 1,000 measurements classified by 500 respondents 
and 100 interviewers is computationally more demanding 
than the analysis of a dataset consisting of 10 groups of 100 
measurements each, classified by 50 respondents and 10 
interviewers. Sometimes it is worth omitting some of the 
observations (measurements in combinations of respon- 
dents and interviewers that hardly occur) to make the 
partitioning more efficient. 

MLn/MLwiN provides some procedures (via the com- 
mands XSEArch and BXSEarch) that are designed for that 
partition (Rasbash and Woodhouse 1996, 89-93). We used 
the BXSEarch command. The command starts an enhanced 
procedure, which attempts to provide the maximum separa- 
tion with the minimum deletion of data. We started with 
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4,790 measurements, 3,026 respondents and 275 inter- 
viewers. After omitting the observations indicated by the 
BXSEarch command, we’re left with 4,597 measurements 
on 3,026 respondents interviewed by 275 interviewers. No 
higher level units (respondents nor interviewers) are left 
out. The procedure resulted in 7 partitions with a maximum 
of 44 cells in the cross-classification of respondents and 
interviewers. The model converged sufficiently fast when 
implied this way. 
The results of the analysis are reported in Table 3. 


Table 3 
Analysis of all the Respondents (s.e. in brackets) 
model a model b model c 
Fixed 
Measurement level 
constant 3.894 (0.136) 3.967 (0.155) 3.864 (0.165) 
year -0.053 (0.055) 
Respondent level 
sex 1.808 (0.153) 
education -1.914 (0.148) 
press1 1.185 (0.090) 
press2 1.197 (0.102) 
Random 
Level 2 
Interviewer 
Os, 2.777 (0.373) 2.716 (0.368) 2.844 (0.363) 
Respondent 
oe} 11.810 (0.635) 11.800 (0.635) 7.017 (0.527) 


Measurement level 
o 
-2 LL 


A df* 


13.130 (0.475) 13.150 (0.476) 13.460 (0.480) 
2TITA 27716.3 27042.1 


Note: * compared to model a 


This table looks very much the same as Table 2 but there 
is an important difference. In the random part we marked 
level 2 — interviewer and respondent to make clear that the 
interviewers do not constitute a third level in this analysis. 

Model a is the null model: no explanatory variables, but 
the variance of the dependent variable separated in a mea- 
surement part, a respondent part and an interviewer part. 
There is a significant interviewer variance. Thus in this 
design we again have evidence for an interviewer effect. 
You have to be careful about the interpretation of the rela- 
tive sizes of the variances if one classification has far fewer 
units than the other (Goldstein 1995, 117-118). It’s not fully 
correct to state that there’s 5 times as much variation 
between respondents than between interviewers, but again 
there is much more variability between respondents than 
between interviewers. 

In the next model (model b) we’ve included the time 
variable (YEAR). Again this variable turns out to be not 
significant and its inclusion does not provide a better fit of 
the model. Again we can conclude that there is no signifi- 
cant overall evolution in “don’t know” answers over time. 
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Model c is the model with the respondent variables. They 
are all significant and this model has a far better fit than the 
previous ones. The substantive interpretation of the para- 
meters is the same as in analysis 1. Women use the “don’t 
know” answer more than men and a higher education 
results in less “don’t know’s. The extent to which the 
respondents follow the political news in the press is also a 
predictor of the use of the “don’t know” answer. The less 
they follow politics the more they answer “don’t know”. 


8. CONCLUSION AND DISCUSSION 


The general conclusions of this article are methodologi- 
cal as well as substantive. 

Our analysis confirms previous research findings about 
the use of the “don’t know” answer. It is related to the 
respondent’s education, sex and a measure of involvement 
or interest in the subject. Furthermore it is likely to diverge 
from interviewer to interviewer. All our analyses showed a 
significant interviewer effect. We did not find a significant 
evolution in the use of the “don’t know” answer over time 
in the two waves of the survey. The interviewer effects 
prove that the “don’t know” response alternative is not 
merely a result of the respondent answering the questions. 
It stresses the necessity of an interviewer training, which 
includes instructions on how to ask difficult questions and 
how to deal with “don’t know” answers. 

As in most panel surveys, the nonresponse in the second 
wave of this panel survey was not totally random. It is 
related to the respondent’s living arrangement, his or her 
political interest and a few socio-demographic variables 
(Loosveldt and Carton 1997). This selective dropout puts 
limits to the generalizability of the results concerning the 
evolution in the dependent variable, but our analyses did 
not show a general evolution in the use of the “don’t know” 
answer anyway. An impact of selective nonresponse in the 
second wave on the size of the interviewer effect is not 
unlikely either as interactions between the respondent 
characteristics and the interviewer effects are possible, as 
analysis 1 showed. But it is unlikely that this will affect the 
substantive conclusions about the interviewer effects. Given 
the results of analysis 1 and the conclusions in the 
Loosveldt and Carton paper, one could even expect that the 
interviewer effect in analysis 2 and consequently the overall 
interviewer effect might be somewhat underestimated. 
Loosveldt and Carton (1997, 1021) show that lower 
educated respondents are more likely to drop out of the 
survey than higher educated respondents and analysis 1 
showed that the interviewer variance is higher for lower 
educated respondents. 

The methodological conclusions consider the use of the 
different models to analyze interviewer effects in panel 
surveys. The analyses presented in this paper show that 
quite complex designs with complicated data structures can 
be analyzed by specifying the appropriate multilevel model. 
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The first model (Analysis 1) only suits in a tiny number of 
cases. It is not so common to ascribe the same interviewers 
to the same respondents for different waves of a panel 
survey, neither is it always feasible. 

The second model (Analysis 2) is an appropriate tool but 
can require enormous computations. MLn is quite powerful 
and helps to decrease the storage requirements, at the cost 
of a small loss of information. Besides, the second model 
has its limitations too. Using this method it is not possible 
to model interactions between respondent variables and 
interviewer variance, as we did in the first analysis, or 
between respondent and interviewer variables. However the 
analysis showed that this model could be a very useful and 
flexible tool. The cross-classified model is also suitable 
when the number of measurements increases. A panel 
survey with 3 or 4 or even more waves, where some inter- 
viewers are retained and some are new at each occasion 
would require exactly the same analysis. The multilevel 
model also knows how to handle respondents for whom 1 
or more measurements are missing, as our analysis showed. 
The pliability of this model outweighs the impossibility to 
include respondent — interviewer interactions in the model. 
That would be feasible when analyzing each wave of the 
panel survey separately. But those analyses could not model 
a possible evolution in the dependent variable, which is 
another important advantage of the joint analysis of all 
waves of the panel. 
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APPENDIX 1 


The rating question was: “Political parties are said to be 
“Catholic” or “non-Catholic”. Please place the cards of the 
various parties on card No. 20 at the place that corresponds 
best to the degree in which the party is “Catholic” or 
“non-Catholic”. If two or more parties are just as “Catholic” 
or just as “non-Catholic” in your opinion, place the cards on 
the same square. If you do not know how “Catholic” or 
“non-Catholic” a party might be, then simply put its card 
aside.” 


With the card: 
Catholic 0 12345678910 Non-Catholic 


The press question was not identical for both surveys. 
For the first survey the press question was: “How often do 
you read the political news in the newspaper?” 


With the response categories: 


1=(almost) always, 2=often, 3=now and then, 4=seldom, 
5=never 


In the second survey it became: “How often do you 
follow the political news on the radio, on television or in the 
paper?” 

The response categories remained the same. 


APPENDIX 2 


Section 4 set the assumptions of the different models that 
were used. For the last model the most important assump- 
tions concern the random effects associated with the res- 
pondent and the interviewer. The assumption that the 
Sconstant Values for the respondent and for the interviewer 
are normally distributed can be assessed by looking at 
Normal probability plots for the residuals. Graph 1 presents 
the plot for the standardized respondent residuals and graph 


2 the plot for the standardized interviewer residuals. 


stand. residuals 


normal scores 


Graph 1. Standardized respondent residuals 


equivalent scores 


by Normal 


In this graph the departures from the diagonal are rather 
limited and no apparent violation of Normality can be 
inferred. On the other hand it is worth noting that this graph 
shows more observations at the upper right hand than at the 
lower left end. 

Graph 2 does not show any clear departures from the 
diagonal either. But in this graph some outliers draw the 
attention. Especially the outlier at the upper right hand side 
of the graph seems to be outside the range of the other 
interviewer residuals. Moreover in this graph also there are 
more observations at the upper right hand side than at the 
lower left end. 
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stand. residuals 


normal scores 


Graph 2. Standardized interviewer residuals by Normal 
equivalent scores. 


The conclusions from these graphs are as follows: there 
is nothing clearly wrong with the residuals but the more 
numerous deviations upwards and the outliers of the inter- 
viewer residuals could possibly be further investigated. 
Efficient techniques for these checks are not yet available 
for multilevel models (Goldstein 1995 29). But it is of 
course possible to analyze a dataset without the outliers. 
That is done in Table 4. 


Table 4 
Analysis of the Dataset Without the Interviewer Outliers 
(s.e. in brackets) 


Fixed 
Measurement level 
constant 3.853 (0.162) 
Respondent level 
sex 1.820 (0.153) 
education -1.929 (0.149) 
press! BY (0.090) 
press2 Aly (0.102) 
Random 
Level 2 
Interviewer 
ihe 2.495 (0.333) 
Respondent 
ove 7.109 (0.530) 
Measurement level 
o 13.420 (0.481) 
-2 LL 26850.2 


For the analysis in Table 4 we excluded two inter- 
viewers, the one with the lowest and the one with the 
highest residual. The coefficients in this table are very 
similar to those of model c in Table 3. The interviewer 
variance has decreased a bit, as a result of the exclusion of 
extremes, but there is no evidence of a considerable impact 
of the outliers on the results. 

The other assumption about the interviewer and respon- 
dent random effects is their mutual independence. The 
interviewer and respondent residuals should not correlate. 
That is of course more difficult to evaluate since both resi- 
duals are connected to their respective units, which do not 
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correspond. You will get 3,028 respondent residuals and 
275 interviewer residuals. An indirect check of this assump- 
tion is possible by attributing the interviewer residuals to 
the respondents. This is done in graph 3. 


-— 


respond. residuals 


[ interv. residuals 


Graph 3. Standardized respondent residuals by standardized 
interviewer residuals 


In this graph, again the more numerous deviations 
upwards and the interviewer outliers draw the attention. 
Apart from that, no pattern can be discerned. Because of the 
interviewer outliers, there are fewer observations at the 
right hand side of the graph. But the respondent residuals 
do not really tend to be smaller if the interviewer residuals 
are higher. Neither is there any evidence of the opposite. 

The check in graph 3 is imperfect as it attributes the 
interviewer residuals to the respondents. A better alternative 
might be to fit a more complex model with an interaction 
term between the two random effects. Goldstein (1995, 
119) proposes this model. A test for the model improvement 
due to the interaction term can give an indication for the 
presence of a correlation between the residuals. Another 
alternative is the insertion of an additional level (the region) 
above interviewers and respondents. That model would 
include a term for the regional variation, which could cause 
a correlation between the interviewer and respondent 
residuals. Snijders and Bosker (1999, 159-160) describe 
this model. But both models require a different parameteri- 
zation with various sets of dummies. Their clarification 
calls for a paper in itself and is consequently outside the 
scope of this paper. 
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Screen Design and Question Order in a CAI Instrument 
Results From a Usability Field Experiment 


MAREK FUCHS' 


ABSTRACT 


Screen design and questionnaire design affect the interviewer behavior in a CAI environment. Previous research has shown 
that interviewers can work more properly and efficiently if suitable functions and features are incorporated in the CAI 
instrument. Usability experiments with the household roster of two large government surveys have shown that using grids 
and tables is an important feature to facilitate the interviewer’s performance. While these experiments were conducted under 
laboratory conditions, we have results from a first field experiment. In March of 1998 a CATI survey on immigrants was 
fielded in Germany (response rate 84%, n = 501). Four different versions of a household roster were compared in this 
production study, testing two different screen designs together with two different question orders in a 2x2 factor design. 
The four versions were randomly assigned to interviewers and respondents. Time measures were built into the CATI 
program, and 234 randomly selected interviews were video taped and analyzed according to a coding scheme. Based on the 
data we assessed the usability of different CAI design features. The results show that the screen design as well as the 
question order have a significant influence on interview duration and interviewer behaviors. Especially the grid based and 
topic based version allows the fastest performance in terms of time used to complete the instrument. Results from the coding 
data suggest that the differences between versions are due to specific interviewer and respondent behaviors. The data 
indicates that the grid based topic version enables a respondent oriented interviewer behavior, and thus allows the best 
interviewer performance in terms of duration. 


KEY WORDS: Computer assisted interviewing; Usability Testing; Field experiment; Screen design; Question order. 


1. INTRODUCTION 
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budgets once the up-front investment for hardware and 


Computer assisted interviewing is on its way to 
becoming a standard survey technique (Couper, Baker, 
Bethlehem, Clark, Martin, Nicholls and O’Reilly 1998). In 
telephone surveys as well as with personal interviews, more 
and more studies are conducted using computer assisted 
interviewing techniques (CAI). Many of the large govern- 
ment surveys in the US are in the transition to CAI or have 
completed it already. Even in Europe, we observe a shift 
towards computer assisted interviewing (Schneid 1991; 
Fuchs 1994, 1995; Laurie and Moon 1997; Projektgruppe 
SOEP 1998) — even though, the methodological aspects of 
this development do not constitute the main focus of 
European research, so far. 

Researchers and people responsible for fielding surveys 
rely on computer assisted interviewing for several reasons: 
(Sometimes it seems, however, that substantial arguments 
are less important than just a specific market rush towards 
CAL.) 


— They hope to collect data of higher quality due to 
built-in consistency checks and range checks during the 
course of the interview. 


- CAI provides the possibility to use automated skip 
patterns and allows to design more complex instruments 
without putting too much burden onto the interviewers. 


— They hope to spend less time and money for inter- 
viewing and post-processing and decrease survey 
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software is payed off. 


— They hope to benefit from CAI’s ability to read external 
data into the interview which is especially interesting 
with panel studies. 


The general movement towards CAI is evaluated posi- 
tively. Researchers and field directors benefit from it 
(Nicholls and deLeeuw 1996) and interviewers (Couper and 
Burt 1994) as well as respondents (Baker 1992), reveal a 
great deal of sympathy or at least acceptance. On the other 
hand, computer assisted interviewing has introduced some 
additional problems into the interview situation, too: in the 
early years methodological research was mainly concerned 
with hardware and software problems (see Couper, Groves 
and Kosary 1989; Weeks, 1992 for overviews). Instead, 
recent studies dealt with interview and respondent accep- 
tance, interview duration, and usability issues (Couper et al. 
1998 for an overview). The present paper contributes to this 
later discussion of “technology effects” (Fuchs, Couper and 
Hansen 2000). 


2. THEORETICAL BACKGROUND 


For the purpose of the following analysis the theoretical 
focus is mainly on two usability issues: (1) segmentation of 
the interview flow and (2) lack of interviewer flexibility. 
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1. Segmentation: in a CAPI environment the interviewer 
has an additional burden: the process of keying takes 
place in the interview situation. Usually, an interviewer 
reads a question, receives an answer, enters the data, 
presses [enter] and then the next screen with the 
following question appears. Compared to PAPI inter- 
viewers cannot look ahead and anticipate the next 
upcoming question while recording the answers to the 
previous one and they cannot start reading the next 
question before pressing [enter] — they cannot work 
simultaneously on both tasks. As a result of this proce- 
dure the interviewer respondent interaction is segmented 
by [enter] keys. So far we do not have quantitative 
evidence that this kind of segmentation harms the data 
or the interview situation. But it is argued that the 
interviewer loses the “big picture”, and the relevance of 
questions and their relationship to each other may be 
unclear (House 1985; Groves and Mathiowetz 1984). 


Our findings from several series of usability tests in the 
lab concerning the screen layout of a household roster 
(Couper et al. 1997; Hansen, Couper and Fuchs 1998) led 
to the suggestion of a specific screen layout that allows the 
interviewer to develop a more complex understanding of 
the instrument, maintain the interaction with the respondent, 
and enter data at the same time: Two different versions of 
a series of questions were tested under laboratory condi- 
tions in terms of the time necessary in order to complete the 
questions and ease of use. We compared a so-called item 
based design with a grid based design. House and Nicholls 
(1988) distinguished between three approaches in screen 
design for computer assisted instruments: item based, 
screen based and form based design. In the item based 
approach one question and one input field are displayed at 
a time, and logic operations are performed in the transition 
from one item based screen to the next. This design is easy 
to program and focuses the interviewer’s attention on the 
actual question. The screen based approach combines 
several items that need to be answered in sequence. All 
logic operations are executed after each item. On a form 
based screen, many items are presented at the same time in 
a table or grid and the interviewer may use the cursor keys 
to move from field to field and to complete them in any 
order. 

The item version tested in our experiment matches the 
characteristics specified by House and Nicholls (1988) for 
a screen based approach. In contrast, the grid based design 
is best described as a form based instrument. It allows 
interviewers to record the information in the order chosen 
by the respondent, it provides the interviewer with a better 
overview of the instrument and it more easily allows up- 
dates and backups (for details see Couper et al. 1997). Also, 
the design matches the interviewers’ demand for more 
questions on one screen — both for speed of administration 
and for context knowledge. The following graph gives an 
impression of an item based and a grid based CAI screen 
design. 


We found evidence that the grid based design reduces 
the segmentation: interviewers could start reading the next 
upcoming question while still entering the data to the 
previous question. Even backing up seems to be easier 
within a grid design. On the other hand, we found only 
modest support for a grid based design in terms of time 
used to complete the task (for details see Couper ef al. 
1997). This leads to the question: what can we do to 
decrease segmentation and to further improve the efficiency 
of a household roster in terms of duration? 


2. Lack of flexibility: The second feature that might cause 
problems in a computer assisted interview is the lack of 
flexibility. One of the advantages of a CAI instrument is 
the fact that an interviewer can hardly skip any 
questions. Although CAI instruments can make exten- 
sive use of skip patterns and filters, they apply a pre- 
defined question order. Usually, each question needs an 
[enter] key before the system goes on to the next screen. 
It is seen as an advantage that this rigid question order 
avoids any trouble the interviewer might have with the 
routing through the instrument, questions for specific 
respondents, filters and skip patterns and so on. He or 
she can abandon this task and focus on the adminis- 
tration of the actual items. On the other hand, this causes 
a very Strict question order and provides the interviewer 
with little flexibility in terms of question order. A small 
example demonstrates this effect: most CAI instruments 
apply a question order to their household roster, where 
all items for one person are asked before the interviewer 
works through the, same items for the next person 
(“person based design” see Couper et al. 1997; Fuchs 
2001 or “grouped questions” see Moyer 1996). The CAI 
instrument, for example, might request the respondent’s 
age, educational level, and other questions first before 
asking for the age of the respondent’s wife. (This can be 
explained in part by the way computer programs and 
data bases work: households represent the main records 
and persons or other entities are treated as subrecords.) 
When completing the questions of a household roster it 
might happen (and in fact it happens quite often, see 
below) that the respondent provides not only the answer 
to the current question (e.g. “I’m 34 years old”) but also 
to a related question: “I’m 34 years old and my wife is 
32 years old” or the respondent might answer “We are 
all Black” when asked about his or her own race 
(Oksenberg, Beebe, Blixt and Cannell 1992). 


While working with a paper instrument it is an easy task for 
an interviewer to make immediate use of the additional 
information provided by the respondent. In case he or she 
answers, for instance, “We are all black” the interviewer 
can easily mark the appropriate check boxes for all house- 
hold members at once. For someone interested in question- 
naire design this leads to the following question: given the 
lack of flexibility in a computer assisted environment, what 
is the best question order for collecting information about 
all household members? 
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Figure 1. Item Based Design vs. Grid Based Design 


Moore and Moyer report results from an experiment on 
two different question orders designed for collecting infor- 
mation about all eligible persons in a household (Moore and 
Moyer 1998a, 1998b). The first question order asks all 
questions for the first eligible person in the household and 
moves on to the next person, when all questions are com- 
pleted. This question order is called a person based 
approach. In the second version, the topic based approach, 
the first question is asked for all eligible persons, then the 
second question for all persons and so on. Moore’s and 
Moyer’s results show strong support for a topic based 
design: the topic version leads to less item non-response, 
less break offs and refusals and is substantially shorter. 
Besides interviewers show significant preference for this 
version. 

In the experiment presented in this paper we tried to 
make use of the advantages of a topic based approach and 
of a grid based screen design: we combined the two screen 
designs (item based design vs. grid based design) with the 
two question orders (person based order vs. topic based 
order) and tested all four resulting versions in a field 
experiment. In doing this, we had the following assumption 
in mind: the usability of a CAI instrument is not only a 
programming issue, but it is also connected to the question- 
naire design and to the interview as a social situation. Both 
aspects of a computer assisted instrument, its screen design 
and its question order, support or hinder a smoothness of 
the interview flow. Based on the results of the previous 
research we had the following hypothesis: The combination 
of a grid based screen design and a topic based question 
order allows the most efficient interviewer respondent 
interaction. 


3. METHODS 


The experiment took place in Germany in March 1998. 
Immigrants of German origin from Poland, Rumania and 
the former Soviet Union were surveyed. Starting February 
28, 1998 and ending March 20, 1998 15 interviewers 
completed n = 501 interviews. All respondents received an 
advanced letter and were called by phone up to 15 times. 


The response rate reached 84% and item non-response was 
considerably low. The interviews were conducted using the 
CATI program CI3. About 95 questions on various topics 
were asked. The average interview lasted 23 minutes. 

Four versions of a small household roster with three 
items per person were included in the instrument: an item/ 
person version, a grid/person version, an item/topic version 
and a grid/topic version. All versions applied the same 
question wording and interviewer instructions, however, we 
modified the screen design and the question order 
according to the theoretical approach mentioned before 
(Figure 2). The item based person version is considered to 
be the standard version — it represents the questionnaire 
design usually applied to socio-demographic portions in 
CAI surveys. One of the four versions was randomly 
assigned to each interview — and thus to interviewers and 
respondents. We measured the total time needed for the 
household roster and in addition the time spent on each 
single item in that section of all 501 interviews. In addition, 
234 interviews were selected at random and the interviewer 
working through the household roster section was video- 
taped. The video segments were coded in terms of inter- 
viewer behavior and respondent behavior and the resulting 
data was combined with the time measurements. 


4. RESULTS 


The durations of the four versions differ significantly 
from each other: interviewers needed 6.6 seconds per item 
in the item based person version (which is considered to be 
the standard one). In contrast each item took 5.5 seconds in 
the grid based topic version. This is a reduction of about 
17% for the grid based topic version. The two other 
versions are in between. 

It is important to mention that both factors seem to 
contribute to the decrease in time used to complete the task. 
If we distinguish between the two factors, we end up with 
the following results: the two topic based versions are signi- 
ficantly shorter than the two person based versions and the 
two grid based versions take significantly less time than the 
two person based versions. The combined effect applies to 
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item/person 


item /topic 


grid/person 


grid/topic 


Figure 2. Four Versions Tested in the Experiment (Each Box Represents One Screen) 


the grid based topic version and leads to the value of 5.5 
seconds per item. (An analysis of variance reveals that both 
factors — the screen design as well as the question order — 
contribute independently to the decrease in time (screen 
design: p<0.01, one third of total effect; question order: 
p<0.001, two thirds of total effect, no significant inter- 
action).) 


seconds 


item + person 


grid + person item + topic 
version 


grid + topic 
p<0.01 


Figure 3. Duration per Item by Version 


But why is the grid based topic version faster? A detailed 
analysis shows that this version is especially faster when 
collecting the information for the second and all following 
persons in the household — a significant impact, that is 
called loop effect (Fuchs 2001). This term describes the 
following phenomenon: the interviewer takes much longer 
to collect the information for the first person in a household 
compared to all subsequent persons. The average loop 
effect sums up to 3.4 seconds per item which is a reduction 
of about 38% compared to the first person (Table 1). 


The loop effect is not specifically characteristic for this 
experiment. We recognized loop effects in our previous 
experiments with the NHIS household roster, too (Couper 
et al. 1997). It is, however, interesting to observe that the 
loop effect is significantly larger for the topic based 
versions than it is for the versions that follow a person 
based question order (Table 1). Thus the topic based 
versions do increase the acceleration for the second and all 
subsequent persons in a household and consequently show 
a larger loop effect. (One implication of our experimental 
design might be that interviewers did not know what 
version they were approaching. This may have decreased 
their performance on the very first item. But this effect 
should be the same across all versions, so the results should 
not be affected.) 


Table 1 
Duration and Loop Effect (Seconds) 
Item Age Arrival Status All items 
Duration per item 
First person in household 9.4 9.9 TT 9.0 
ee 6.6 5.7 4.5 5.6 
All persons SOnes Hasire Galan Teehetts 
Loop effect 
Differenz between first 
and all other persons in 
the household -2.8 -4.2 -3.2 -3.4 
Loop effect by version 

Grid + topic -6.8 -6.4 -4.6 -5.9 
Item + topic -5.2 -8.3 4.3 -6.0 
Grid + person -0.5 -2.7 -3.0 -2.1 
Item + person 0.3 -0.3 -1.3 -0.4 
Average loop effect =D Othe -4,.2** 3.25% -3.4*** 


**p < 0.01; ***p< 0.001 
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Analyzing the video tapes we can provide reasons for 
these differences at least in part: given the topic based con- 
ditions, both interviewers and respondents adapt differently 
to the interview situation compared to the person based 
versions. When asking the questions for all persons in the 
household, the respondent recognizes the logic of the proce- 
dure very quickly. In quite a high proportion of all cases 
(about 30%) their reaction to this is “We all arrived in the 
same year” (meaning: “Don’t ask me this question again 
and again”). 

If the instrument follows a person based design, the 
interviewer has to memorize this piece of information, and 
if it comes to the next person, he or she needs to remember: 
“Do not ask this question again, the respondent gave you 
the appropriate answer already!” Only in a few cases they 
really do, most of the time they just ask the question again, 
This is especially true when using an item based screen 
layout that gives no clues in terms of the answers to the 
same question for the other household members. In a topic 
based design instead, the interviewer can easily adapt to 
that situation. Thus he or she just enters the same code for 
all persons in the household without asking the question 
repeatedly. Both the interviewer and the respondent get 
used to the questions, and so the question answer process 
runs with less verbal contributions from the interviewer’s 
side as well as from the respondent’s. Both interviewer and 
respondent can anticipate the next question and the inter- 
view runs more smoothly. This is especially true when the 
CAI instrument makes use of a grid and provides further 
context information, e.g., the responses for other household 
members to the same question. (Looking at the results 
reported in the lower part of Table 1 we conclude that the 
grid based person version does not benefit to the same 
extent from the advantages of the topic based approach. 
However, due to the grid design the loop-effect is 
considerably larger than in the item based person version.) 
As a result the time used per item is substantially shorter 
and the interviewer can provide respondent oriented 
interviewer behavior similar to Schober and Conrad’s 
(1997) findings. 

Providing feedback by the interviewer sometimes works 
as a signal that he or she has recorded the answer to the 
previous question in order to stimulate the respondent, so 
that the latter guesses about the next question and reveals 
the appropriate answer even without an additional stimulus. 
In extreme this might lead to a respondent behavior where 
he or she provides the information about all persons in the 
household at once: “We all came in the same year”. The 
different versions tested in this experiment impel and 
support such behaviors to different degrees. From our 
results we can conclude that the grid based topic version 
stimulates interviewers and respondents to deviate from the 
scripted interview to a higher degree than the other 
versions. As far as duration is concerned this version allows 
the interviewer to make efficient use of information 
provided for all household members at once. Evidence from 
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the video coding support our interpretation of version- 
specific occurrences of time saving interviewer behaviors 
(1) and respondent behaviors (2): 


1. By means of analyzing the video tapes we observe quite 
a lot of interviewer behaviors that do not follow standard 
interviewer procedures: besides the fact that about 78% 
of all items are read as worded, interviewers do not 
administer 9.3% of all items to the respondent. In 
another 5% of instances, the interviewer does not read 
the question but instead provides a different stimulus 
containing the relationship of the next person to the 
respondent (e.g., “... and your wife?”). (It is interesting 
to recognize that interviewers chose the same verbal 
expressions on their own that Moore and Moyer 1998a, 
1998b scripted in their experiments on question order.) 
In 5.5% of all cases the interviewer does not read the 
question but rather verifies the answer (“... and your 
wife is 32 years old?’”’). Some incomplete questions and 
wrong fills are observed, too. In total we have about 
22% of all items affected by at least one interviewer 
behavior that does not follow a standardized interview 
script — which is a surprisingly high value considering 
that all interviewers were aware of the fact the 
interviews were video taped! Compared to other studies 
on interviewer behavior, however, the values are 
considerable lower. For example Oksenberg, Cannell 
and Blixt (1996) applied behavior coding to the National 
Medical Expenditure Survey and reported 37% to 41% 
of such interviewer behaviors. We will come back to the 
question of whether or not these behaviors help obtain 
valid measurements. 


We draw the following conclusion from these particular 
findings: most of these behaviors indicate kind of a short- 
cut, e.g., the interviewer does not read the question text as 
worded, he or she tries to make the conversation smoother 
and more suitable in terms of conversational rules. From 
our point of view this indicates that interviewers do not 
want to ask for information the respondent provided 
already. They do not want to behave unresponsively toward 
the verbal contributions of the respondent, instead, they 
wish to follow conversational rules. As a side effect these 
behaviors are less time consuming than standard inter- 
viewer behaviors. In our perspective, the priority therefore 
lies not with saving time, but with customizing the question 
answer process to respondent behaviors not anticipated and 
not absorbable by the computer assisted instrument. 

In order to compare the four screen design versions in 
terms of the degree of interviewer deviations from the 
standard interview script we have computed the proportion 
of items per case affected by this kind of behavior. Large 
differences in interviewers not following the scripted inter- 
view between the four versions are to be noticed: Applying 
the grid based topic version to an interview results in more 
than twice as many such behaviors (the average proportion 
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of items affected is 0.48) than the item based person version 
(0.21) which is the standard for most studies so far. (An 
analysis of variance indicates that both factors contribute 
independently to the overall effect (screen design: p < 
0.001; question order: p < 0.001; no significant interaction 
effect). About 25% of the overall effect can be attributed to 
the screen design, about three quarter to question order.) 
And this contributes to the time used for interviewing: items 
affected by a interviewer behavior not scripted in the inter- 
view take substantially less time (4.0 seconds) than the 
regularly administered items (6.8 seconds; p < 0.001). 


Table 2 
Interviewer Behavior and Respondent Behavior by Version 


Grid+ Item+ Grid+ Item+ 
5 ; Total 
topic topic _person_person 

Average proportion of items 
affected by interviewer 
behavior not following the 
scripted interview percase 0.48 043 0.34 0.21 
Respondent provides 
information for all persons 
in the household at once 


**D < 0.001 

In order to differentiate between the proportion of cases affected by 
a certain respondent behavior and the average proportion of items per 
case (!) affected by a certain interviewer behavior we used percent 
notation for the first and decimal notation for the later. 


OBGzAS 


"38.2% 44.4% 29.0% 10.8% 29.7%*** 


2. Additionally an analysis of the respondents’ behavior 
shows that the topic design leads to a higher proportion 
of cases (42,3% compared to 19.7% for the person 
approach; p < 0,001) where the respondent provides at 
least once in the household roster section the informa- 
tion for all persons or a group of persons at once (e.g., 
“We all came in the same year”; “We all have the same 
legal status”). By contrast, the difference of the grid 
based design from the item based design is considerably 
smaller (33,6% vs. 26.1%) but does not reach the level 
of significance. However, an analysis of the interaction 
reveals a significant interaction effect (p < 0.05): Using 
a topic oriented question order the grid design does not 
make a significant difference. However, on top of an 
topic oriented question order the grid design increases 
the number of instances where the respondent provides 
the information for all household members at once. 


It is surprising that results differ even for the two screen 
designs when using a person oriented question order. The 
study was administered by telephone, the respondents not 
being aware of the screen design at all. The only possible 
explanation is based on the fact that the interviewers modify 
their behavior in concordance with the screen design, stimu- 
lating the respondent differently. Accordingly, respondents, 
as well as interviewers, react to the screen design and the 
question order under the grid based person design in a way 
that facilitates the interviewer respondent interaction and 
thus helps smoothen the interview flow. (As seen before, 


the interviewers change their behavior even under the grid 
based person condition (Table 2), however, the question 
order does not stimulate respondents to behave accord- 
ingly.) 

One possible drawback of these interviewer and respon- 
dent behaviors might be a lack of data quality due to 
changes occurring in the predefined question answer 
process; instead, the respondent considers the answer less 
intensively and thoroughly. We observe only very few item 
missing values so an analysis of this standard indicator for 
data quality is not efficient. In fact, we do not expect a 
higher proportion of item missing values in either version. 
One might, however, be concerned about the homogeneity 
of the answers provided by the respondent. In a high pro- 
portion of cases he or she listens to the full question text 
only once and that could contribute to a less thorough 
consideration when answering the same question for 
subsequent household members. Additionally, answering 
for all household members at once (“We all arrived in the 
same year’’) might increase the homogeneity of the response 
and thus decrease data quality. 


Table 3 
Average Number of Different Categories (Homogeneity) 
per Household by Version 


Y auanie Grid + Item + Grid + Item + Total 
topic topic person person 


peanetanivele® (ete Fol oy Wise yee 
categories) 
Status (4 categories) 1.3 13 ih.) 1.3 1.3 


No significant differences 


In order to assess this possible drawback we computed 
the number of different response categories chosen by the 
respondent on a particular item for all household members 
(e.g., for year of arrival: respondent 1985, partner 1987, 
daughter 1987, son 1988 = 3 different response categories). 
This should give us an idea of whether or not only those 
respondent make use of the short-cut (“We came all in the 
same year’’) for whom this is actually valid, or whether even 
other respondents provided one answer for all household 
members even though they should have chosen two or more 
different response categories because of the situation in 
their particular household (unfortunately we have no 
external validation for the responses provided). In looking 
at the average number of different response categories 
(Table 3) we do not notice any differences in terms of 
homogeneity of data. For the year of arrival as well as for 
the legal status (as a German or a foreigner) there is no 
visible difference between the versions. For all versions the 
average number of different response categories chosen 
(one for each person in a household) shows no significant 
difference. 

These finding provide only weak evidence that a grid 
design does not harm data quality. Other standard data 
quality indicators need to be assessed with larger data sets 
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in order to decide whether or not data quality is affected. 
However, based on the data available, we are unable to 
proof a effect on the validity of the responses. 


5. DISCUSSION AND CONCLUSION 


Our results from a comparison of four versions for a 
household roster (using the same question wording across 
versions) indicate that interviewers as well as respondents 
perform more efficiently under the grid based topic condi- 
tion than with the other three versions. Combining a grid 
based screen design and a topic based question order 
reduces the average duration by about 17%. Two thirds of 
this reduction can be attributed to the question order, 
approximately one third to the screen layout. It is important 
to mention that the effect of the screen design is less 
pronounced than the one of question order and — compared 
to the effect on duration — even smaller on interviewer 
behavior and respondent behavior. 

Even though the effects of the grid design on interviewer 
behavior and respondent behavior are far from large, they 
help to elicited two reasons for the better performance of 
the grid based topic version in terms of interview duration: 
(1) in the grid based topic version, the interviewer as well 
as the respondent adapt better to the logic of the question 
answer process, both anticipate the next question more 
easily and the question answer process runs more smoothly. 
(2) This version leads to more occurrences in which the 
respondent provides the information for the persons in the 
household faster and more often the respondents reveal the 
information for all household members or at least for one 
group at once. Even though the results are not fully consis- 
tent, this particular version makes it easier for the inter- 
viewer to adapt to this situation, record the information and 
stimulate the respondent to give the next appropriate answer 
without repeating the full question text. 

Our findings contribute to the discussion of how to 
design survey instruments for interviewer administered 
computer assisted data collection. Based on the results 
reported in this paper we can draw the conclusion that 
making use of grids facilitates the interviewer respondent 
interaction and helps speed up data collection. Our experi- 
ments on item design vs. grid design conducted in the 
University of Michigan Survey Research Center’s usability 
laboratory have shown that we can improve interviewer 
performance by providing grids (Couper et al. 1997). 
Moore and Moyer (1998a; 1998b) have demonstrated that 
One can improve interview efficiency by switching to a 
topic based question order, too. The present paper indicates 
that the interview situation benefit even more when com- 
bining both features. 

Using grids and a topic based question order causes a 
greater amount of instances where the interviewer deviates 


_ from the scripted interview. From a rigid methodological 


point of view this might be seen as an important drawback, 
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especially, if the interviewer deviates from the standard 
interview script using global questions for all household 
members. For example, Martin (1999) showed a significant 
increase in the number of people enumerated in a household 
if extra questions were asked. In addition, Kindermann and 
colleagues (1997) demonstrated for non-household roster 
type questions, that additional cues on victimization signifi- 
cantly increase enumerations of crimes. Generalizing these 
results to global questions across persons, one would expect 
a decrease of data quality as interviewers use non-scripted 
behaviors that apply global questioning methods. However, 
the results reported in this article do not indicate that inter- 
viewers are using global questions and the author does not 
recommend to make extensive use of global questions when 
designing a survey instrument. Instead, the findings lead to 
a screen design that allows interviewers to make use of 
information reported when the respondent switches to a 
global mode and provides the information for several 
household members at a time. So, we do not want to 
encourage researcher to make extensive use of global 
questions and we do not want to see interviewer modify the 
scripted questions in order to ask global ones. However, 
when confronted with a respondent providing more infor- 
mation than actually asked for, the screen design of the CAI 
instrument should not prevent interviewers from making 
use Of it. 

A grid based design has been proven to facilitate the 
interviewers job with respect to this task, because it allows 
interviewers to adjust their behaviors in concordance with 
general conversational rules. Basic findings of behavior 
coding suggest that interviewers frequently deviate from 
specific interviewing procedures. “These changes often 
reflect adjustments made by the interviewers to meet the 
exigencies of the situation: to melt it more congenially with 
communications immediately preceding it, or to adjust to 
the respondent’s particular situation” (Oksenberg et al. 
1992: 3). This is especially necessary when respondents do 
not limit their answers to the information requested by the 
question, but elaborate it or provide additional information. 
“Avoiding the appearance of not paying attention to the 
respondent, interviewers in this situation frequently filled in 
the answer themselves without asking the question, or 
asking it only in part” (Oksenberg et al. 1992: 5). They thus 
try to switch to more respondent oriented procedures to 
avoid looking unresponsive. A grid based screen design and 
a topic oriented question order supports interviewers to 
interact according to these conversational rules and with 
respect to the interview situation’s needs. This might be 
acceptable or even preferable as long as we are talking 
about factoid questions and as long as these interviewer 
behaviors do not harm data quality (e.g., leading question 
or probes). 

What needs to be done in order to improve the computer 
assisted instrument in its supporting function for the inter- 
viewer respondent interaction: Our data suggests that the 
grid based topic version leads to a specific interview flow, 
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so that interviewer and respondent can easily adapt to it. 
Jeff Moore (1996; Moore and Moyer 1998a, 1998b) has 
shown that interviewers prefer the topic based version. By 
contrast, we know little about the respondents’ satisfaction 
with that question order. Assessing their opinion about the 
different version is consequently an important goal. More- 
over, we do not know whether this version matches the way 
in which information is stored in the respondents’ brains. It 
might be, that respondents can easily adapt to this version, 
but that in terms of cognitive and social burden or in terms 
of correctness of answers it is not the right method. Addi- 
tionally, we need to focus on the question whether or not 
we can transfer our findings from a household roster to 
other segments of a questionnaire. Right now we are 
conducting a series of field tests comparing different design 
solutions for factoid information other than household 
roster information and for attitude items. The versions 
tested in this experiment differ in the degree of contextual 
information provided to the interviewer while administering 
a particular item (previous questions, next questions efc.). 
The general question sounds: what happens if we use grids 
or form based screens more extensively? Under what 
conditions and circumstances does it help to improve 
interview efficiency and what are the limitations to this 
approach? However, it is too early to present any results at 
this time. 

In addition, there are more unanswered questions that 
need to be addressed in future research. Personally I would 
like to suggest a specific approach to assess these questions 
assuming that computer assisted instrument design is of 
importance to different clients: researchers, interviewers 
and respondents. Of course, it is important that a CAI 
instrument meets the researcher’s needs to obtain his or her 
measurements and also that the question answer process be 
well designed for each single item. However, in my view 
considering the social dimension of the interviewer respon- 
dent interaction and the behaviors in between single items 
is also a matter of importance. If the CAI instrument 
disturbs the social dimension of the measurement process 
it might harm even data quality. So far we do not know 
which approach allows the best compromise between 
validity and reliability of the measurement process on the 
one hand and a smooth short and non-embarrassing inter- 
view flow on the other hand. In order to find out to what 
respect a specific CAI screen design might harm data 
quality and how it helps save time, money and interviewer 
effort we need to conduct more usability studies. 

To assess the questions mentioned above we do need 
more field experiments. Due to the fact that we want to 
analyze the social dimension of the interview and its effects 
on interviewer behavior as well as on interview duration, 
laboratory experiments do not meet our needs completely. 
Of course laboratory experiments allow a more controlled 
setting, reveal more detailed information about both partici- 
pants, and — as a result — need smaller numbers of cases. 
Still, without going into the field, we will never confront 
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our prototypes and design solutions with real pressure to 
maintain and facilitate the interviewer respondent inter- 
action and the question answer process at the same time. 
Usability testing should therefore be seen as a joint process 
of laboratory experiments and field tests. 


ACKNOWLEDGEMENTS 


Some of these results were presented at the SMP Brown 
Bag Seminar, Institute for Social Research, University of 
Michigan on May 21, 1998 and on the occasion of the 54" 
Annual Meetings of the American Association for Public 
Opinion Research, May 16, 1999. Special thanks go to the 
interviewers participating in this experiment. Mick Couper, 
Siegfried Lamnek, Jeffrey Moore and two anonymous 
reviewers provided helpful suggestions on earlier versions 
of this paper. 


REFERENCES 


BAKER, R.P. (1992). New technology in survey research: 
computer-assisted personal interviewing (CAPI). Social Science 
Computer Review, 10, 145-157. 


COUPER, M.P., BAKER, R.P., BETHLEHEM, J., CLARK, C.Z.F., 
MARTIN, J., NICHOLLS, W.L., and O’REILLY, J. (Eds.) 
(1998). Computer Assisted Survey Information Collection. New 
York: Wiley. 


COUPER, M.P., and BURT, G. (1994). Interviewer attitudes toward 
computer-assisted personal interviewing (CAPI). Social Science 
Computer Review, 12, 38-54. 


COUPER, M.P., GROVES, R.M., and KOSARY, C. (1989). 
Methodological issues in CAPI. Proceedings of the Section on 
Survey Research Methods, American Statistical Association, 
349-354. 


COUPER, M.P., FUCHS, M., HANSEN, S.E., and SPARKS, 
P.(1997). CAPI Instrument Design for the Consumer Expenditure 
(CE) Quarterly Interview Survey. Final Report. University of 
Michigan. 


FUCHS, M. (1994) Umfrageforschung mit Telefon und Computer. 
Einfiihrung in die computergesttitzte telefonische Befragung. 
Weinheim: Psychologie Verlags Union. 


FUCHS, M. (1995). Die computergestiitzte telefonische Befragung. 
Einige Antworten auf Probleme der Umfrageforschung. 
Zeitschrift fiir Soziologie, 24, 284-299. 


FUCHS, M. (2001). The impact of technology on interaction in 
computer-assisted interviews. (Ed. D. W. Maynard, H. 
Houtkoop-Steenstra, N.C. Schaeffer, and H. van der Zouwen). 
Standardization and Tacit Knowledge: Interaction and Practice 
in the Survey Interview. Wiley (forthcoming). 


FUCHS, M., COUPER, M., and HANSEN, S. (2000). Technology 
effects: Do CAPI or PAPI interviews take longer? Journal of 
Official Statistics (in press). 


Survey Methodology, December 2000 


GROVES, R.M., and MATHIOWETZ, N.A. (1984). Computer 
assisted telephone interviewing: effect on interviewers and 
respondents. Public Opinion Quarterly, 48, 356-369. 


HANSEN, S.E., COUPER, M.P., and FUCHS, M. (1998). Usability 
Evaluation of the NHIS Instrument. Paper presented at the 
Annual Meeting of the AAPOR, St. Louis, MO. 


HOUSE, C.C. (1985). Questionnaire design with computer assisted 
telephone interviewing. Journal of Official Statistics, 1, 209-219. 


HOUSE, C.C., and NICHOLLS, W.L. (1988). Questionnaire design 
for cati: design objectives and methods. (Ed. R.M. Groves, P.P. 
Biemer, L.E. Lyberg, J.T. Massey, W.L. Nicholls, and J. 
Waksberg). Telephone Survey Methodology, New York: Wiley, 
421-436. 


LAURIE, H., and MOON, N. (1997). Converting to CAPI in a 
Longitudinal Panel Study. Working papers of the ESRC Research 
Centre on Micro-Social Change, 97-11, Essex. 


MARTIN, E. (1999). Who knows who lives here? Within-household 
disagreements as a source of survey coverage error. Public 
Opinion Quarterly, 63, 220-236. 


MOORE, J.C. (1996). Person- vs. Topic-based Design for Computer- 
Assisted Household Survey Instruments. Paper presented at 
InterCASIC ‘96, International Conference on Computer-Assisted 
Survey Information Collection, San Antonio, TX. 


MOORE, J.C., and MOYER, H.L. (1998a). ACS/CATI 
Person-Based/Topic-Based Field Experiment — Final Report. 
Center for Survey Methods Research, U.S. Bureau of the Census. 


MOORE, J.C., and MOYER, H.L. (1998b). Questionnaire Design 
Effects on Interview Outcomes. Paper presented at the Annual 
Meeting of the AAPOR, St. Louis, MO. 


MOYER, L.H. (1996). Which is better: grid listing or grouped 
questions design for data collection in establishment surveys? 
Proceedings of the Section on Survey Research Methods, 
American Statistical Association, 986-990. 


207 


NICHOLLS, W.L., and de LEEUW, E. (1996). Factors in acceptance 
of computer-assisted interviewing methods: a conceptual and 
historic review. Proceedings of the Section on Survey Research 
Methods, American Statistical Association, 758-763. 


OKSENBERG, L., BEEBE, T., BLIXT, S., and CANNELL, C. 
(1992). Research on the Design and Conduct of the National 
Medical Expenditure Survey Interviews. Final report. Survey 
Research Center, Ann Arbor, USA. 


OKSENBERG, L., CANNELL, C., and BLIXT, S. (1996). Analysis 
of Interviewer and Respondent Behavior in the Household Survey. 
U.S. Department of Health and Human Services. AHCPR No. 
96-N016. 


PROJEKTGRUPPE SOEP (1998). Funktion und Design einer 
Erganzungsstichprobe fiir das Sozio-oeconomische Panel. 
Diskussionspapiere des DIW, 163, Berlin. 


SCHNEID, M. (1991). Einsatz computergestutzter 
Befragungssysteme in der Bundesrepublik Deutschland. 
Ergebnisse einer Umfrage. ZUMA-Arbeitsbericht 91/20. 


Mannheim: ZUMA. 


SCHOBER, M.F., and CONRAD, F.G. (1997). Does conversational 
interviewing reduce survey measurement error? Public Opinion 
Quarterly, 61, 576-602. 


SUCHMAN, L., and JORDAN, B. (1990). Interactional troubles in 
face-to-face survey interviews. Journal of the American Statistical 
Association, 85, 45-54. 


WEEKS, M.F. (1992). Computer-assisted survey information 
collection: A review of CASIC methods and their implications for 
survey operations. Journal of Official Statistics, 8, 445-465. 


rae syns 


: en sd ae 


 « 


PVA 


ae eh 
shernacetiatl 


tt a #4 


sci asia pi Mateo 
‘ pavlovat be, oer 


’ 


epitope 


ienayce athe 


ge wegen 
ce Raised ne RN rae oD 


nie oe hw Mall UGH. 


rediolbne warner atch eget eapedh 
et Pet atipiy = “ely: iS 
; ‘ AA, RE H ' 
Me oA » ah led ie “hey. ‘Sib 
ees TOA sage ee Ay MESON 


isa a Ae - ated aR a 


agA PAGO, 


i ta 


Je FPR 


en Dee ra epithe mrRans 
ee ee ae ah eros? we ci lit ate , 
ri as ve" vi bi dae“ gaitie, 
Yel. uy, 5 Be is yi tas: 
a ashi Nerets ey ts we J 
s er a ma pe i, 
i SANE Wah rave 7? ma vcd yaks Oral K nite 
Ltr lod aoa e a} sat Ny (Oh SEIT Date od, MARIOS aN RY 
; 


Brave 
a 


i se 
’ 4 “A + aA Pa U 
4 : ’ 
ww ‘.- a! Ta 
< i ja es aL tes iy i 
+ i 1 . 
i Ne ee a bay Ue " Tphed i ahah ca ae aie io} 
; ey bry ' ee | 4) | be We ; H i " ne - 
; h| 
e es ’ 4 bah ain’. he iy by hs a svi ‘a ena 
J w) , ’ Ras iu 
/ + i ' er . 4 ewe ’ we Nw ‘ ent 
an ; ( f bad Sa Pay) ’ ue ’ Ours 
v ( ls 4 i. op eth 9 this 3 ia ue 
1 : ny f vd if ; 
; - ad q as { Mea ee} ae 
‘ bps “via Vp Y , 
i es ne fn Shale 1 ae 
Re ee ee ‘I id le ss (rd 
names has 4 "7 fi ae wm pu a ie ‘hy ine 
-F * t Nnte a, * ih pd } 
r erin f 4% OOF aoe ” bei al Wie rT) ; ta 
ig ‘ ’ Pid i ’ 
2 y he \ i ‘ } br rl eae ae ae 15 Si sa ea dh dy w rh rf Li Ngo » ivf 
tok ; s . i wih &) ST Rae Oe ee tom Aye Witten ne 
‘ } ; , ; eat = oh 
. i is a a ae an. hia f' : A, 
é } bs 4 i re h o¥ ay A MY ri 4 } vi 
: i We pas ; iil) ' n 
' : 1 ogiitulld ’ ish) a be i ; ved a t - Le ieee ees 
haa ae ad es. PI a iat a Sl ; 
oh oe a re ks 7h, vi if , ti i A % ult at y ‘a 
yi ‘ " wa . : : iuiasiiy ’ : ate ik a ; hh { i, MH, a yy Ag 
a ,. nat ” i ; ; ‘ion Nh 
c. , sie Peay ty ity apd irre 4 vee vam | ca has yj hy a] 0 "ey ey hithe ; ey i) vb de Pia wie 
ip (iis ‘ VET ni iy fey t 
SY BAUER FEC IT Ui WER Ney ASR a Srna) ie Net i) Pe bhai Ls iyi ARN Hdl ne Pb 
* rtd base | paw y Sad ; ana { hi ' 
WAY Aaa - vars nl COMMS ‘ie hea sa eee fi maha i 
! 7 mt f A Trae j ‘y 1b, 
Hob Yea put ek eM eee al ‘a a ih ne 
ere snk aullk apainetc Ne ats Di CARAON ACE TAD pale PATEL eat esl vee 
} : P " ( Di "i ‘ Hie 
aris Wike.+ ee eis mea: Me 4S cae eh ees eee a vo td rok ae 
es ! ! ; 
re: tia uf P; J b Art's 
gly Leah i Ara Sl asia conntiiend) bie aaa 
: ‘ t . uw y ¢ 
ae: i F 7 Hoh 
\ ; “ j 
: A i 
; ’ vi ry? } F i 
b rs ae i Sor 


209 


ACKNOWLEDGEMENTS 


Survey Methodology wishes to thank the following persons who have served as referees during 2000. An asterisk indicates 


that the person served more than once. 


J.-F. Beaumont, Statistics Canada 
W. Bell, U.S. Bureau of the Census 
* D.R. Bellhouse, University of Western Ontario 
Y. Berger, University of Southampton 
P. Biemer, Research Triangle Institute 
* D.A. Binder, Statistics Canada 
D. Cantor, Westat Inc. 
P. J. Cantwell, U.S. Bureau of the Census 
R.G. Carter, Statistics Canada 
J. Chen, University of Waterloo 
C.Z.F. Clark, U.S. Bureau of the Census 
M. Cohen, National Center for Education Statistics 
* J.-C. Deville, /nstitut national de la statistique et des études 
économiques 
* P. Dick, Statistics Canada 
* J.L. Eltinge, U.S. Bureau of Labor Statistics 
* W.A. Fuller, Jowa State University 
J. Gambino, Statistics Canada 
* M.A. Hidiroglou, Statistics Canada 
D. Holt, Office for National Statistics, U.K. 
J.-S. Hwang, Academia Sincia 
* D. Judkins, Westat, Inc. 
* G. Kalton, Westat, Inc. 
S. Kaufman, National Center for Education Statistics 
J. Kim, Westat Inc. 
* P.S. Kott, National Agricultural Statistics Service 
* M. Kovaéevi¢, Statistics Canada 
M. Kramer, U.S. Bureau of the Census 
P. Lahiri, University of Nebraska - Lincoln 
N. Laniel, Statistics Canada 
* M. Latouche, Statistics Canada 
P. Lavallée, Statistics Canada 
S. Linacre, Australian Bureau of Statistics 
* S. Lohr, Arizona State University 
* H. Mantel, Statistics Canada 
P. Merkouris, Statistics Canada 
J. Moloney, Statistics Canada 


G. Nathan, Central Bureau of Statistics, Israel 
D. Norris, Statistics Canada 
J.-S. Pischke, Massachusetts Instituteof Technolory 
D. Pfeffermann, Hebrew University 
N. Plante, L Institut de la Statistique du Québec 
N.G.N. Prasad, University of Alberta 
* B. Quenneville, Statistics Canada 
* E. Rancourt, Statistics Canada 
* J.N.K. Rao, Carleton University 
* L.-P. Rivest, Université Laval 
K. Rust, Westat Inc. 
I. Sande, Telcordia Technologies 
N. Schenker, University of California - Los Angeles 
F.J. Scheuren, The Urban Institute 
A. Scott, University of Auckland 
J. Sedransk, Case Western Reserve University 
* J. Shao, University of Wisconsin - Madison 
* M.P. Singh, Statistics Canada 
R. Sitter, Simon Fraser University 
C.J. Skinner, University of Southampton 
R.T. Smith, National Agricultural Statistics Service 
E. Stasny, Ohio State University 
* D. Stukel, Statistics Canada 
A. Théberge, Statistics Canada 
* Y. Tillé, Ecole Nationale de la Statistique et de l'Analyse de 
l'Information 
S. Tremblay, Statistics Canada 
C. Tucker, U.S. Bureau of Labor Statistics 
* R. Valliant, Westat, Inc. 
J. Waksberg, Westat, Inc. 
M. Wendt, Statistics Canada 
K.M. Wolter, National Opinion Research Center 
C. Wu, University of Waterloo 
* W. Yung, Statistics Canada 
* E, Zanutto, University of Pennsylvania 
A. Zaslavsky, Harvard University 


Acknowledgements are also due to those who assisted during the production of the 2000 issues: J. Beauseigle (Dissemination 
Division) and L. Perreault (Official Languages and Translation Division). Finally we wish to acknowledge C. Corbeil, C. Ethier, 
C. Larabie, D. Lemire and G. Ray of Household Survey Methods Division, for their support with coordination, typing and copy 


editing. 


= 


Ulvaet, iM ne 
aL ahs aa dapat meheou eet, Coole goneleient bus on ted nach i Ve NE 
a Fe AY vaqnchetnon het ia vigste wah sco iealibbaiia prs? inate sid eal fob ce on 


Veit he 
MINI UA 


. a 7 fi ' yy » 

* seloceintn nae Yael 

y Linn) Ween eget oy ey 
b. f ae Lou =) eae 


headin ‘oie P ed 


i ; 
6) oy wie : 
! oy Wins as 

a ign ‘ pa ! 


| gh 2 (sha eT Wg sported ae ope , 


+ 


Ce Pre) Ae 7 3 
‘ \s ofasicaeth, Bt suansriels 0K i AD tae a : 
wit? deel nett eal Weragbo® » ele a 
vores “eave he eared 1 eh, x ae 
ee ey | Agnid. Bars a ; 
of Tremenhia) * vp weigh Ro ae ae ‘" 
; Pe a Aya ‘ Woe Renate Os ki as 
pre'= adie Aner inet \owiolA lhl’ “F Ata 
dies av) T HN, AD elle . re , 
au be eae We date c be J 
one} rahe aeesaey A : vila Aa 1’ sh n vie vibra 
ihe Vs SSA 8 osha WO ae VRC AE, a , Ne Ente A 


toory |” = | al fu ls na wie 


3 voce, pAbreeTS alae ad Ss Fi basin 
\ Lv Wy nthe, ¥'. erat be 1) ar. Br wil a =m ty ! Y Mr arbi 

A a ue Petite shea on 
“sar Seth eben ek Te Ae wate aS as 
Awe) bee, bled Yi Lae . + i pectin | 
elitngl. spades vn Woo Sc UA tly ae ; (shu jetes oly ~ 

aaah. yp. weave ar aia riba wares van 

te f ot! (ier Als Ate 4 Ww — 
edt iets nstwonX, Ee yi 
feat y wre (Tae é i i , 


‘ ] hy J 


ek ely VR | Sr cacts nevis o ov teal Sih yin a OTM ag" 


’ 


o : f : Wed, " ee ie : iy 
») ' oo we al Pel | ve} 
xing f (i aon , nn 
ee : i : 
eller walk he f 
me iy a) 
Aas ie ie. 
DH he 
4 4 | 7 Veal ads 1 
\ i] sy | a ak aay 
HN tn i 
; ‘ j Mf feet ; ; 
( F f ; a 1 ah (4 a4 it abs 
; ‘ Ny y bi i i 5! ia i, 
7 é uf eis ' 
gti Pe Maes | 
iF wy if iNT eee Lee j 
p se ne sa a Watiy : 
, 4, ty ny alle ‘ 
. i Pu Vier ? i | q Li 
1 ! 28 7 ae et 7) 
eae | i a4 “a my, i i ie rf A ee - . 
“ A 7 ,S ? + < a! he tau tce eA if i i - 


The Canadian Journal of Statistics La Revue Canadienne de Statistique 


CONTENTS TABLE DES MATIERES 
Volume 28, No. 2, June/juin 2000, 225-448 


James O. RAMSAY 


Differenviallequationy models forstatisticalifunctionsteavytr rte ree eer «as deceive oe sisrels ose ole alecse Uytecn town OL 225 
Nancy E. HECKMAN and James O. RAMSAY 

enalizedireeressionawithamodel-based penalties err rerwtt-ulcaa Pacey wart ecisiefiee ts cock sare tere cabs ition aierh Siete 241 
Debbie J. DUPUIS and David C. HAMILTON 

Revressionmesiduials and teststatisticss assessing naive outhierdeletionian seem eee eee ee eee ee ai ene ieee 259 
Martin BILODEAU and Pierre DUCHESNE 

Robustiestimation olutents WRamOGeLI i rer anete ores ore cricieueraraleye oui ores sy ab Srccuceolc enc sstapelsfoepabspsacteyoMNOR hoi Rak css Bevan 277 
Joris PINKSE 

Nonparametric two-step regression estimation when regressors and error are dependent ................ 00 ec eeceeeees 289 
Ursula U. MULLER 

Nonparametric. regressioniforthreshold data c.carycncrtors tate one oars ater apt re niente bio et ancielereln ate la aw eames we ace ih omevas 301 
Ronald W. BUTLER and Aparna V. HUZURBAZAR 

Bayesian prediction olewaitn oatimesun) SLOchAaStiCiMOelS nae rer wepinreiel utes eieiate teat nies eau ctets eae kel wenn Lede ened oie Siu 
Caterina CONIGLIANI, J. Ivan CASTRO and Anthony O’7 HAGAN 

Bayesian assessment of goodness of fit against nonparametric alternatives ..... 0.2... cece cece ee ence eee ene 327 
Caterina CONIGLIANI and Anthony O’ HAGAN 

Sensitivity onthe tmactionaliBayesdactor tojprio;n distributions). wes ielci cle item eie nde wet aslo) tele aie aie eet sys 343 


William J. REED 
Reconstructing the history of forest fire frequency: 


identifying hazard rate change points using the Bayes information criterion ........... 0.6. cee cee cee cee tee eee e eee ees 353 
Antonio CUEVAS, Manuel FEBRERO and Ricardo FRAIMAN 

Estimating. the number OfClUuSters ptr ciecil- eich haven clef hone se) see ctetevehsscunys oh-=ts,< le ieaiehcdol Meee roles CBAC tors on nate ON Mentos eee reer 367 
Peter T. KIM and Ja-Yong KOO 

Directional mixture models and optimal estimation of the mixing density ........ 0... cc eee c eee eee eee t eee eae 383 
Dominique FOURDRINIER and Idir OUASSOU 

Spherically symmetric distribution with constraints on the NOPM .... 6.065. oe oe ee eee cee ete tees meee see nee 399 
Serge B. PROVOST and Young-Ho CHEONG 

On the distribution of linear combinations of the components of a Dirichlet random vector ..............2++++e eee sees 417 
Michael D. deB. EDWARDES 

Implications of random cut-points theory for the Mann-Whitney and binomial tests .......... 2.2... eee eee eee eee eee 427 
Martin R. PETERSEN and James A. DEDDENS 

Effects of omitting a covariate in Poisson models when the data are balanced ............. eee e eee cece eee e ee eens 439 
Konstantinos FOKIANOS, Amy PENG and Jing QIN 

‘A-genetalized-moments specification test for the logistic link? ......0.0.2 65 scess cess cons eset dence oncmecahe ane 446 


Horhcoming, Papers AruCles a) paraltle myareyer- terete cleteiciersuesste: vel te eredete che eiedeneeeMtis ier ele) ove) otoksl} sv shel -VeleNov aioe) #) eel oieueeleneraiareie!s¥ 447 


The Canadian Journal of Statistics La Revue Canadienne de Statistique 


CONTENTS TABLE DES MATIERES 


Volume 28, No. 3, June/juin 2000, 449-672 


Feifang HU! & John DS KALBELEIS@H:theestimating functionibootstrap mee ito tite ere ree 449 

Discussion: 

James V..ZIDEK. & Steven Xs WANG: Comment 1! v.05 eis wove at onze othe teins «ute eee lates atone nee vere reine er ret 482 

Thomas J. DICICEIO & Robertd TIBSHIRANI; Comment 2). cacnece iin erect einer te eerie erent 485 

Christian LEGER: Comment 3 (0, 3.5 50si4d.1 sartedale tp iimap eeieeamberass sepicclinie dent bae sheer epee eres 487 

Angelo'J: CANTY. & Anthony €; DAVISON: Comment 4. Wi...) <j otteirel sls eitolers s aktionachtel she PUR as trei ikuea eet ae eee 489 

StephensMzSS LEE Comment Si yite wets tie ce) sie erekenauoieve seekers bin sews erste eae teusiiehs ackanscin Laren Repo eae Sec hor teteeror er vce 494 

Rejoinder: 

Feifans HUi& John Ds KALBFLEISCH) iva cajecca thes oye o sicrccsr stare hts areca eckeevoL este ces etescoaicr Mette Eat Rta ie ome Tomei e e Eetcas 496 
Alan M. POLANSKY: Stabilizing bootstrap-t confidence intervals for small samples ............. 00. cece e cece eect eee eens 501 
Steven E. STERN & A. H. WELSH: Likelihood inference for small variance components ........... 00. c esse cence seen eee nee 517 
Francesca DOMINICI, Giovanni PARMIGIANI & Merlise CLYDE: 

Conjugate analysis of multivariate normal data with incomplete observations ............ 0... cee eee eee eee ee eee ene 533 
Barbara TONG & Kert VIEEE? Smooth estimates ofnormalmixturesi cee vas ioe er ep aon esate eet lteter 551 
Dianliang DENG & Sudhir R. PAUL: 

Scoreitests:ifor zero inflation ingeneralized. linear modelsy cme acc cela 1 iceeraty tee i eye eaete ere 563 
Jiti GAO & Thomas YEE: Adaptive estimation in partially linear autoregressive models ............ 2... ccc e eee eee eee e eee eee 571 
Thomas S. FERGUSON, Christian GENEST & Marc HALLIN: 

Kendall’ sitau:for serial dependence... nacieje dace ceed tee, soenckerece bee «nla geoepere ave Sial vc ouspeld al ac ek ened ea eke gl nee RT 587 
Min CHEN & Gemai CHEN: Geometric ergodicity of 

nonlinear autoregressive modelsiwithichanging conditionalivantancesw an. etre reise yeild een tl tere eee era 605 
Jean-Yves DAUXOIS: Inférence par les martingales pour 

dés processus ponctuels a'compensateur discontinue sas ite crete ce etree ne ei ka et te eee ee 615 
Bing LI: Nonparametric estimating equations based on a penalized information criterion .......... 0. cc eee eee eect eee ee 621 
Peter Xue-Kun SONG: Monte Carlo Kalman filter and 

smoothing, for:multivariate/discreteistateispacemodels/siym aac sictoneten crete hak. toner et onscreen aval eaten pat teen RSet ete Panera 641 
CheikhvAwl, DIAGK: Surlaiconvergence'desitestside Schiee etide Yatchews yy. -1)-1ierie eto crate create ecto teae an wee era 653 
Forthcoming; Papers/Articles:a paral tie rrncra oieucieyoretexs sferatareiei Soke’ Musneve ston cisrst amsusicr Gite ds slay cust nietelcPel tefols cei sicre ee me aC eae area mode 671 


Volume29 (2001);Subscription\rates/Braisi dfabonnementiers wreaaucy-p yee ay xcnstegs ohtu-k niet doaspal aebail scicieiis ee eta eee 672 


JOURNAL OF OFFICIAL STATISTICS 


An International Review Published by Statistics Sweden 


JOS is a scholarly quarterly that specializes in statistical methodology and applications. Survey methodology and other issues pertinent to the 
production of statistics at national offices and other statistical organizations are emphasized. All manuscripts are rigorously reviewed by 
independent referees and members of the Editorial Board. 


Contents 
Volume 16, Number 1, 2000 


Correcting the Bias in the Range of a Statistic Across Small Areas 


DaVIGiRRO UA KINS SONA UI TU wane reteare Se ENN Te TR RUTTER oT ee Ee ee Ee te en 1 
A Procedure for Stratification by an Extended Ekman Rule 

Date.) COLTS REGS Sts ISDE OO eS TOO Ce OT eee Ra LN Oda ea bah ee tas MAM A ALS lee 15 
A Note on Use of Inverse Sampling: Post Estimation Between Successive Infections 

LOY ed (ORE AG IGS oh se, iat We Nathan PG ACE ort Ono ie Sees Ia AS aE cee ES ae ese eC ee 31 
Optimal Weighting of Index Components: An Application to the Employment Cost Index 

WO RRAN SI GAIT YTING C1 GI: ) FOR RAL AN RS BIA ARO GET SO CEO ORTOP OR CEO ORO IT COMI 0S rd Lt CO A Aca las 39 
Random Selection in a National Telephone Survey: A Comparison of the Kish, Next- ae and Last-Birthday Methods 

WD ONEEDINS OMe CSSCHAM CUNCROIGNANG) OSCD IAs CALAniay vrs st et ae eee ee ee ee eee eee 53 
The Effect of Different Rotation Patterns on the Revisions of Trend Estimates 

[BOG SHIA! CHEN GRO GOI (D) B01 4 Perini s Re A PE NCRORs OOOT PUTO a SON TOR ROO PO eae ee RTS Oss 61 
ES GONTANGIS OLLWALE! REVIEWS tote te hte ise orcs oie Oe she im amie asan ace sie ai a atn wlohememeeses sosbten lara 's|teweuls Gna! eg Shere buchen share) acoreints 2 excwoka Mare cnsenee as id 
TMC T/T OUEM AIS renee ye ce ese ote eseyes tall ass 15, sies 6 star Dunn a's anh ai Gh ovonapedesia sl -Oinl aisha. HATTA Mar anede lS 9 ese) sus lore lake ee: Bue eunueteloiees a8 Seer peeee 83 

Volume 16, Number 2, 2000 

Recent Developments for Poverty Measurement in U.S. Official Statistics 

WAV GNVISBClsSONMGONSLANICCAR CLE O TARA RODEHT Ie MIC ROCIA Ret ea teen tie eae TUTOR OUST re aoa eee cacetcee ote ey reo atts scoala puters sTae 87 
Nearest Neighbor Imputation for Survey Data 

IGRUGIEREMGNG JUNISAAOP tity Se A Ne ee TL ANT PER Seiten fst Anca Be dee saieyens «i = ial> ets 113 
A Note on Jackknife Variance Estimation for the General Regression Estimator 

LEV TEM UCHESTIC, Ma antares eT oe Ie) eRe aL uae as Mist aval Pome Necks Bin SHMToWe Ream hee colds Sohek vale S PE eee 133 
Stratification by Size Revisited 

Allasa Jet, IDaspiern CaO NORE VOUHADE Bissc 08 on boo ED OSECORS 6 omU bbe don odooo sn pou Hoos an DONO Gude osm UddenTe DOD eS 139 
An Estimation File that Incorporates Auxiliary Information 

CaryiulelsakieviMalkedawHs Usaysand Wayne A. Husler. <j eas ells cette ie heya eee lee oie 155 
Large Scale Fitting of Regression Models with ARIMA Errors 

Byonnbischermand ChristophesRlanas: snc sacine hee ee - 22) yoo «ots ole: 6 Lees) shoo ete neked riled edocs) = hay Moly ote ge aye aisvedode yeast“ ee) -led- Its} 


BS OOKSANIG SOLWAFE IREVICWS she erin nc ic ne Sis mea elie ne hae Sea Vr ueMnuevalie) dace ote olla avadevaa evn) Anite ete ayouenectenete oie RS eee 185 


Volume 16, Number 3, 2000 


Model-Based Alternatives to Trimming Survey Weights 


Michael: R.-Elliott.and Roderick: J As Littles seo ee re eee ee oe Ant ne hoe eae or Cn ne cee eae 191 
Permanent and Collocated Random Number Sampling and the Coverage of Births and Deaths 

Lawrence R Ernst, Richard Valliant, and) Robert J) Cassady, man ere tne eee ee oe eee ree ee 211 
Survey Estimation for Highly Skewed Populations in the Presence of Zeroes 

Forough Karl berg, ossveseie a cpsreueiciousies oieree lle 6 5 Sefo aban aio etch MAO TGA: SOD Laie Sle A AG Slo ieuketoiae ie, Ses OUR EA etn nee eee ere 229 
The General Application of Significance Editing 

David Lawrence:and Richard McKenzie Vey eterno otoia. spel onle «levers Os fe ees estes to ete ee ere he heerlen pee Ieee eee ee 243 
Developing Usability Guidelines for AudioCasi Respondents with Limited Literacy Skills 

Sid J \Schneider‘and Brad EQWGrds® 23.86.4046 site tee On ale WTR GORI ile tanece ols Grol Patent TOR PEATE ae nae SCRE 255 
Technology Effects: Why Do CAPI Interviews Take Longer? 

Marek Fuchs, Mick Couper:‘andiSue ‘Ellen Hansen nicen ton oven bt 8 oe ee oe hei oes a OR Oe Se SR ee ae eee 273 
Book and Software Reviews’. 3/sc).G) 2g ek ee «ois a oe ose) ola vee boy aie Oe alee meee eho lohisunte Unusie Sexe muctlaceteastens eLepeenetet si aetan el Manatee 287 
Editorial: Collaborators... ... 5: is inds:.o\e.. & ile, oieire josie sovescota ssn sce oneal eeu ool Lins egos ole uous yo PMO RA Es oes PR ATS EM TORN ERGL SUo Nee 291 


All inquires about submissions and subscriptions should be directed to the Chief Editor: 
Lars Lyberg, R&D Department, Statistics Sweden, Box 24 300, S - 104 51 Stockholm, Sweden. 


GUIDELINES FOR MANUSCRIPTS 


Before having a manuscript typed for submission, please examine a recent issue of Survey Methodology (Vol. 19, No. 1 and 
onward) as a guide and note particularly the points below. Accepted articles must be submitted in machine-readable form, 
preferably in WordPerfect. Other word processors are acceptable, but these also require paper copies for formulas and figures. 


4.1 


4.2 


oy) 


Bed 


Layout 


Manuscripts should be typed on white bond paper of standard size (8% x 11 inch), one side only, entirely double spaced 
with margins of at least 1% inches on all sides. 

The manuscripts should be divided into numbered sections with suitable verbal titles. 

The name and address of each author should be given as a footnote on the first page of the manuscript. 
Acknowledgements should appear at the end of the text. 

Any appendix should be placed after the acknowledgements but before the list of references. 


Abstract 


The manuscript should begin with an abstract consisting of one paragraph followed by three to six key words. Avoid 
mathematical expressions in the abstract. 


Style 


Avoid footnotes, abbreviations, and acronyms. 

Mathematical symbols will be italicized unless specified otherwise except for functional symbols such as “exp(-)” and 
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Short formulae should be left in the text but everything in the text should fit in single spacing. Long and important equations 
should be separated from the text and numbered consecutively with arabic numerals on the right if they are to be referred 
to later. 

Write fractions in the text using a solidus. 

Distinguish between ambiguous characters, (e.g., w, @; 0, O, 0; 1, 1). 

Italics are used for emphasis. Indicate italics by underlining on the manuscript. 


Figures and Tables 


All figures and tables should be numbered consecutively with arabic numerals, with titles which are as nearly self 
explanatory as possible, at the bottom for figures and at the top for tables. 

They should be put on separate pages with an indication of their appropriate placement in the text. (Normally they should 
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after the reference, e.g., Cochran (1977, p. 164). 
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should not be abbreviated. Follow the same format used in recent issues. 


‘$]U9991 SOIQUINU S23] SULP VSTINN J[QPOUL 9] SIAING “SUO] Ne S}LIO9 3NQ JUSAIOP SONAII ap Sat SY] ‘UONKoI[qnd ap sguur | 
B O19 ‘9 ‘q ‘B saa] Soy JURINOfe US sJUU UIQUI 9UN ,p Jd INajNe SUIQUI UN,p suONRoI|qnd so] JonsuNnsiq ‘onbrsojouosys 
2IPJO Ud 919 JUDATIOP INoINe UIQUI UN _P san sat 1a onbyaqeydye sspslo usd ag WOp 31x93) UN .p uly e] & BIYdesSoI[qiq eT] 
(p91 “d *LL61) ueIy90_ :o]duiexq 

“Q0udI9I9I e] Sgide a]janbe] Janbrput ‘99319 1sd JUDUINSOp UN ,_p ated 

aun Ig ‘uoNvorqnd ap ayep P| 19 sInajne sap WOU 9] JasIogId JUDAIOP 3}X9} 9[ SULP Sd}TeJ XNVALI} SON _P B SIOUIIIJOI SIT] 


arydeisonqd 


‘(stoJ o1gtuasd vy] Inod souasgjas Wey A Inb asessed np said saigsul ang JUDATOP S]I “USWIOTPULION) 

‘2]X9] J] SULP IOINZIJ JSAIOP S|I NO WOIpUs,| sp UONeoIpUT ouN JoVIOd 39 sogsedgs sased sap Ins oNrered dAIOp SI] 
‘(xne2]qe) Sop jnevy Ud 39 SaINSIJ sap seq ne) ajqissod anb yneorydxa 

Issne 91} UN J91I0d 39 SaQLIL SOIJJIYO SOP DIAV JIINIISUOD JIPIO UD SJOIOUINU 31}Q SNO} JUDATOP xNeIIqQe} SI] 19 SOINSIJ SIT] 


XNvII qe} 39 SIANSIY 


"9}X9} 9] SUBP JURUBI[NOS 

9] ua onbryeyt usa gurtiduit ang yop inb 90 Janbipuy ‘s}oul sap ImJossai osrej Inod sgsijnn yuos sanbiyei sasgjoesed say] 
‘(1 ‘150 ‘OQ ‘0 ‘@ ‘M SUTUIOD) SNBIquIe $d1Q}9RIEO Sd] JUDUOITE]O Jonsunsiq 

‘anbiyqo o1eq UN _.p apie | & 9X9} 9] SUP SUOTIORIJ So] BILIOW 

‘uIo] snyd aouarayaq Wey A INayNe | 1S aOIp ey e oqere aIyJIYO UN Ied JyNoQsuOD 

dIpiO Ud sag}OI9UINU 39 [edriouLId 3)x9) np sdaredgs ang JdAIOp sajueIOdUIT 39 sonsuo] suONeNbe say ‘ajduits soedsa un 
B BIPUOdSaIOD JUSATOP 91X29} JI SUBP SdI1Q}9KIVO Sa] SNO} seu ‘jedroutid 3)x9) 9] SUBP IOINSIJ JUDATOP Sd1INOD Sa[NULIOJ Sa] 
‘919 (-)3o] 39 (-)dxa suTUI0D sjauUOTOUOJ 

sojoquiAs saj Inod jnes ‘olteNUOS UOTROIpUI oUN.p suTOUI e onbiyeyW Ud souMtIduT jUOIJOs sanbnewsyyeu sooquiAs sa] 
"Sa[SIS Say Jo SUONeIAgIQe Sa] ‘saded sap seq nev so}OU Sd JOA 


uoNsepIy 


“QUINS9I 3] SUep sonbreUTSYyyeUL 
SUOISSOIGX9 $3] IOWA “S9[D SJOUI XIS B SION ap IAIns aydesesed un.p gsoduioo guins9i uN Jed JIDUDWIWIOD HOP 93X92} IT 


guinsay 


‘atydessor]qiq e] Japgooid STeUI SJUDUISIOIOUIOI S93] DIAINS Op sxouUe 9jINOT, 

"2)X9} Np uly eT & oINTeIed JUDAIOP SJUDUISIOIOWIII Sa] 

‘2}x9] np o8ed osgruaid ve] ap seq ne 9}0U oUN SUP JoINSI] JUSATOP Inajne onbeyo op assaipe | 19 WIOU FT 

‘sotidoidde san sap yueyiod sa9}0J9UINU SUOT}O9S Ud SYSTAIP 31}9 JUDAIOP $d}Xx9} SIT 

“INON 1NO} Jonod Z| SUTOUI Nv Pp SaBILUI Sap dav 39 INOYed DUSI]JO}UI B]Qnop eB 

‘JUSUA[NIs 9dvJ OUN Ins ‘(saonod | | Jed 74g) prepue}s yeuLIOJ op ourj{q Jorded un ins sgrydesBo]Ajoep ag JUSAIOP $9}X9} SIT] 


uoIjejUaseag 


cs 


LS 


(ay 


Iv 


Ce 
le 


"SOINSIJ SOP 19 SO[NULIOJ Sap JUDIE) 3] 
Inod 9931xa ssoye vias Jorded Ins UOISIOA OUN sTeUr ‘satqeidadoR JUOS S[SIOIZO] SAN Sa] “JOdJIagGpJO AA JUSWIa[quayoid ‘9}xo) op 
JUSWIITIET] OP SIOTYOIJ IP SULIOJ SNOS STUINOS 31}9 JUDAIOP Sa}dad0e Saforye Sa] “sNOssap-19 sjurod say Ja}0U op 19 (| ,U ‘6] ‘JOA Np 
Inued e) ajanbua, p sanbiuysay, ap ju2991 OJQUINU UN JoUTUeXa .p aJQtd ‘ameuINos 9] Inod 91x9} aNOA JatydesZopA}Oep op WRAY 


SALXGL SAd NOILVINASAad VT INVNUYAONOD SHAILO AIG 


“USPIMGY “W]OYYIOIS 1¢ POI - S ‘ONE HZ XOg ‘UspeMmsS sonsHEISg “uoUedsg Gay ‘B19QGK7] sIET 
-JOUPT JOYD Oy} 0} pojasIp aq p[noys suonduosqns pur suorsstwgns jnoge sainbut [Ty 


CaM tel MoV Meet RPA Note owek Way sae wain ah eee vicy ay «5! ss, 5, 3c lolinite “alle bovine ple deitatasttoneits, «tiara: eieie atten ae Meee liar ecteuel shomenate Ghote $10} €10qUIJO? [eLIONIP| 
SAE 10-8 D400 FOS Ue Egy Ode 6 Boro DONO EAD GIO oy In Miata paid a Crore aco Nimo 6.5 ae Oke ata 6 oleae SMAIAIY IVAIJOS pue yoo 


eRouscle tei eusucelte weiss lol saisarei ol le/hei tral elie) dlrs.) b/i4i 56) (9/6) 1o1)0 <ei.6) wire’ oa. \0)\8 a) e]\e\'4! s1.0' 6) 6)-0) vie), ieee elie! e) (01 a! (#,.0 (8) § (oe uasuoy] ual” ang pup ‘““adnoy YOU ‘syon YadDN 
(JasUOT OY], SMOTAIOU] [qVO OC AYUM :S}00]Jq AZSojouyoay, 


5ig:6 B38 sO 0G ODS 5D Ob ROOTS jor OPT THOS ERO AAD OR NERCE 5 ro nL ne ORO It PRE One gt Rae PEA. eh ic Re SpaoMpy poag pun saplauyos ‘f PIs 
STIPIS Aoeloiry powuiry yim sjuopuodsay Isejoipny Joj saulpapiny Ayypiqessy Surdopsasq 


2. 0-day oun dg HeOtO ch Gucn OLR Ee Seo: OCS eR ear e OR CaER anery CharT tr DieRee hen rmR EC Cae, Sisinge aIzUayoy PADYOIY PUD BOUaLMDT pIAavg 
SUNIPY soueoylusis Jo uoneoryddy [esousy syL, 


ue heel heel suena elie: cele autsire) e) a) lel te le 'ei 0/16) >! (ele .6) "0 © 0llw\4, (9) ‘6 19' 1¢\ 0-6) (0 1a) |e). 6, eet @eMaliel aie tener ie! 0, le (ee) 0.10. 6 ie Sie! © \o))8) yeu. 07.0) 10/\e ie) (01a. (ew silr (ele) Oe ens, 0, B12Q [AD Y Yysnosoy 
S9O0ISZ JO UaSIIg JU} UI sUONe[Ndog paMmoys A]YSTY JO} uoHewNsy Aoaing 


Ohehne hee. 9) le) nod eles eel $0.0) \6. as). 60),9 1/9) &).8..0\ 16) a\76: 18,6. :0) 0) 9. (4 ce, 9:10 pee), 0/0) 0, .¢ 1b) 016 19) 0, S$) 0, #16 eive:.6 8 Aposv) [ 14aqoy pun qUDI;JDA pAoyoiy YSUAT Y JOUIAMDT 
syjeoq pure surg Jo o8eJOAO_D oy} pue Sul[dues JoquINN WopueY poeo0][OD pue juoUeULIDg 


JiSOCARL SHON BAU GRO God, OOO RON REPRE ON RINT eee een eee Pac oo apy “Ff yoldapoy puv NONI TY [avyoyy 
SIYSIOAA AVAING SUIWIWLY, 0} SOANVUIDITY pose g-[spoyy 


0007 ‘¢ 49quIny ‘g] auIN[oA 


FS) SMOIAIY BAMIJOS pue Yoo 


€LI ec 0 6 oe eles si ew ieee 6 © eld «le! «0 ele, Ce. 516 too) a 16) 6 Bike (4 16..6;/6\ eienielie’ 0) e; ay © .s\fe tar s),¢ je! "6 16/8 (68116; 6.16) a) enters. (ele: Pile) | 118 spud] d aydojsiay) pup ABY ISI usolg 
SION VAIUV UUM S[OPO| UOIssaIZOY JO Bu Veo oBIe'T 


SSI 4, [8) #,"¥ (ele) 9, te Me bie) elie elie) oe) e/\eile cave 19:!¥)ce).¢) 9) lelael iar oie ecw coils) (6) '« soil) (9)iw/uy’ 9) (¢))8) \¢/ #176) © fale) 18/15) sia) (elie) Aal{ny v audn yy puv ‘KOS ], HL ‘Dpay] WW ‘LYDST “Ii auoy 
uoneuojuy Arerixny sayesodsoouy yey) of] uoneuNnsg uy 


6£1 €16 @ 0) 6. ww 0 ie ay @ Ol Oe ore 9 Ob) 6 Gel ig \eile..0. 1014) 046 Oe) 16.18 61°61. 6) by Bie) -@ 010, 6.010) .0 0) Siena © « ote 1s) Ke) #618 6 /ecm 81s) @ wee) junio, péoyoty puo upufiog ‘HH uvp 
PoustAdy 9zIg Aq uoneoyneNsS 


1 aUusayong asda 
JOWUINSY UOIssoIZOyY [eJoUayH oy} Joj UONeUINS| soURLL A IJIUPoes UO DION V 


CLL EEE SEED ent n ttt nner n ett e ete e ene nn ney opys unr pup uay) onyvir 
eyeq AdAINg JO] uoNe Indu] JoqysIaN IsaTeaN 


Te J2DY 1p ‘J, 14aqOY PUD ‘OAjI) ‘xf BOUDISUO ‘UOsIag ‘WW piADq 
SONSNEIS [PIOYJO “SA UL JUaUIOINSeaA] ALAA, 10} syUaUIdO[sAaq JUD90y 


0007 ‘7 49quiny ‘gy auINjOA 


€8 S pee 6 ves Sew see 6m le wale + 6 0 pie ee ee viele an @ Rie ee sie o's © 0 e/slelein Si exe \eiieiie ‘eller #16) 6/19) b.¢)'9 oiieiiel.e 4 etbie! phe) 6) viiel is») «16lelel elie) @ieusie sjeuinor 2YO uy 
LL Ste; ce) 'w lokere: (6! b. W)'9 ee leie, oe. m eeerd & 4) .0) 190 C9, whos bs le Fie. SO bind, 6 [exe 'e site e ey be my oheh) e's] efi! suelo (e's. © ie¥y,.< el ee, julie. s’ s\te le 9l'wie)\a/ pew SMIIAIY IIEMJJOS pue yoog 
19 Pe) \\8 (6ii0; e "0. w.@ ce 6.0) 0) 8) 0 ee ee wba Ose) be w F Aiea elle te e:(e) ©) ai\e; ae. Colle wile (sfel8) 4a) 010s Ve iptiel Wm) eile «16 ohevierie (ese levers uadv TOW H 3104) pun 1241S 79) plavq 


SOJLUINSY PUI] JO SUOISIADY OY) UO SUJANeg UOTLOY JWoIITJIC Jo 199IJq OL, 


€S CO COM Sat at eC fat Lat aC tat ht tet yet Bat JAC Ta ed Ie Ome PAL ete CM Dd oC i MOP RO ee eal) Ce ae CCEA Pee Cli tee net WA De pIunjvD A ydasor puv ‘pjoyouv) Vv assar ‘uosulg auviq 
spousal Aepyyig-isey pue ‘Aepyuig-3xen ‘Yysry ey) Jo uosuedwiog y :AdAIng suoydejay [euoHeN & UI UOTaTIg WopuLY 


6 ETE eee ener tenet ett e tenet tees ulajsuamaoy ‘f yAvpy puo noNeT “y avyoyy 
XOpu] Iso_D JuswIAo[dwgq oy} 0) uoneot{ddy uy :syusuoduioD xapuy jo BuNysio~y ewndo 


PS i eRe alte i a le a a tae Tae Pata Hes RATS eRe ca Cece d ree ene ge a INT Suor-Suny 
SUONDIJU] DAISSADONG UdeMIOg UONLUINS| 3SOg :3ut[dures asIaAU] JO VS~) UO DION V 


eee unpapy ug 
g[ny ueUILyY popus)xg ue Aq uoNLoyNeNS 10} aINps00Ig VY 


[EEE ene eee enna nv] ung pup surypne -y piavg 
Svaly [[eWIS ssoloy oNsNeIS & Jo adueY oY) Ul seIg oy) BUNIILIOD 


0007 ‘I 49quinyy ‘g] auINjoA 
s]u9}U07 


‘pieog [RUOUpY oy) Jo s1oquiowl pue sooJaja1 JUapuadapul 
Aq pamataAal A[Snososu ase siduosnuew |] I[V ‘poziseydurs ose suoneziuesio [eonsneys Joyo pue saoijo euoeu ye sonsieis Jo uoNoNnpold 
9) 0} JUSUTJJOd sonsst JoY}0 pue AZojopoyjour Asaing ‘suoneordde pue ASoopoyjowl [eoNsNeIs Ul saztfe1oods yey Ajowenb Apsepoups & st SOL 


UIPIMS SdsHVeIS Aq pIysyqng MILAdY [euoyeUsIUy UY 


SOILSILVLS TVIOMAO AO TYNANOL 


(71 juoulauuoge ,p steL4/sovel UoNduOsqng :(100Z) 6Z SUINIOA 


IL9 eR PL ee es Sees mets! ol wiie i bie. wee 16: (s).61_61 861.0) ,6/ eos he) lene «vie, sm ake ew 6 lei 8, m6 18) e656 16> O! a6 ee © 4. e, fae 6 anariele diene oryered R soponty/siodeg SUMWODY04 
€S9 Dae bie ale wie 6 ine jelaielce is) a bist ele 0.0 0: 6-6 610 6. 0 0 16 6:0 .0 © «.b 0 0.68.) 010 MOYOIE X op i) aaTYoS op $}$9} sap 3J0UdBIOAU0D el Ins “MOVIG if ‘Vv yyloyp 
1v9 fe (esoue tel bre at bye) ie ele eq) tale, ob el 61016) 0) 0.6) obec lee eee 6 whe 0-6 « © © .ele: 6! 6) 0 9 0) 60) eee 0%, s[opour goeds 9781S 9]0I9SIp oyeWeAN[NuL Joy SUIYJOOUIS 
pue Jo}[1J UeWTTey OpeD aIUOW ‘ONOS uNy-enxX JoIEg 
CQ ena PT ar RF eee PEG te LIS IRIESD) Sher HS 4% UOU9}LIO UONeULIOJUI pozijeuod e uo paseq suonenbs Sunewinse otoureieduon ‘J Surg 
S19 CRO Sh OWen siLel w elh) o Kel e,19)\a 7s) b a76;'s, le ef si 16.6) 616.0 18) 0.001 6 0.0 6.0.16 10'¢ 10) > 16 e610 1018) .6) © 10!0 el ol el eel e es nunUOSSIP Inoyesuaduioo e sjemouod snssoo01d sop 
Inod soyesuryreul sa] Jed sousI9JUT ‘SIOXNV SoA X-ueor 
S09 Siete menemotadatelitel steeds tdisiets delet sessctelielels) s) els. 'so/.e! allele kalle,scs| elie oe SooURLIvA [euonrpuos Zuisueyo UJI STopour SAISSOIZOIO NL yeourfuou 
Jo AjoIpoSsa OLeUIO0SD :NAHO FUlseyH 27 NAHO UW 
L8S Pee wie eet ws in isl ste) < ecalie leds =) Cine (6.8) Ba: 6 Wa) 0 6 618 0b 0 6.6 8) Bd. 6 @ Re 16 8 ee 0 ee 6 oem ee Ow fw Bele) elie a oun late douspuodap [euos JOJ ne} S$ [[epusy 
‘NITTVH 2RW 2 LSANAD UeNnsuyd ‘NOSNOUA 'S sewoyy, 
WY acaba enna Ray Pg’ oe thE See enol e 8 so. Fs S[EPOU dAISsoISaIONV JeoUT] AyeNsed ul uoNeWINss sANdepy :qyA SPUOULL 7 OVD UL 
GLE 00 PECTS SERGI AOD a EE ESE RA ea ce a s[apoul JedUT] pozipesoues Ul UONLIJUI O19Z JOJ s}s9] a109_ 
"TNVd ‘UY Wypns 2 ON sueyueig 
ISG coche eee eee t een e teen ees SOINIXTU [PWOU JO sa}eUUTISa YIOOUIS :ATAIA U2 2 ONOL weaeg 
SS Gama ny cabernet op selec PngeMe trae erie ses, 56 eee" sy SUONBAJOSQO 9}o{CUIOOUT YIM BJep [eWUIOU BJeLeANNUI Jo stsAyeue sJesnluoD 
‘ACATO PSO 29? INVIDIN Vd MueAolyd ‘TOININOG e9seours{ 
EUS etoile cl eketetielis) #enel| elie ele is) efeteve 6 41% #114) 6.6.6 16. 6: (06) 6, 6 sjusuodui0s QOUvLIvA [jews Joj Q0uUdIOJUI pooylfoyry] ‘HSTAM ‘H Vv 2» NUALS a udA91S$ 
NRE PB SEES BGS ics i Ae Oa a sojdures [Jews 10J speasojut souapIUCS j-de1s}00q ZuIzIIqeis -AMSNW10d ‘W Uelyv 
96r MUeieiels is elletieirel} she ile\ cel ete) slice. 9.0) 9.) 9 ¢..8' 08,  0)o.16 6.6, 6 (000 6 6:8) © ©: 8 ¥)e © 18 B96 0) 6.6 0 e818 8 82 je 2 a8 @ 8 HOSIFD TT IVS ‘qd uyor wy ANH SULTIO] 
Jdapuiolay 
P6r Seonsdatictciicls calls ni cis lelcicicclicdsMolsie ic, cpenie Siaiie) mieies eles asic lsLeneuei stele yeeeusiceie re veLeuelelehe ieee agape gnasciele in Collen uedieiieze c JUSUTWIOZ Ri ig | “9 W uoydais 
Cy Ei IED a SAIS Se a ee Re a p JUSWIWIOD :‘NOSIAV ‘DO Auowuy ® LINVO ‘f ojesuy 
Pete ite rete aire icra (ates sie a viele st= ciniste vten tinn se a'vmle vio Riagrione ten eH eleir rnin wa ias € UOUTIOD “Y_ORT wensuyD 
3 Z 1USUIWIOD :|INVUIHSALL ‘f Yeqoy 29 OIDDIDIC “f SewoUy, 
Gogg 0 oa oat ao Can a a | 1UaWIWOD :ONVM *X UAAelS 2 WACIZ ‘A soures 
/uoissnasiq 
Sapp C8 SE PBS Ae RE ETS Eee aI Se a dems}00q uonouny Suneunse OUL ‘HOSIATAGTVN ‘c Uuor 2 NH Zuryio4 


ZL9-6bh ‘0007 UInfjaune ‘¢ “ON “BZ BUINLOA 


SHAPLLVW Sdd ATAVL SLINA&LNOO 


Oooo 


anbysyezs sp suuatpeuey anaay eT SoSIVU}S JO [VuANOL ULIPEURD 94 


Ltr 8 0) @ fs le),0) 0:0) a; 6). © 10119. \€).0, #0114 16/050: e020 eset) elie) 4) (0) 66! evie \e (e,,6.18 6 80 else) ss. min) ee de fonvel.n ie! vie. eley/eie)(e)isre (a\tei's)e) 817s, ojrered R soporty/siodeg surmosyyo,j 


PRN sonar Beamon hl) gtot cae Lhe so: XUr aNs1B0] ayp 105 1891 uoNRaYIOAds syuauIoW-paztferaUEd y 
NIO 3urg pue ONAd Aury ‘SONVIMOA sounuejsuoy 

Gop cherie hau sais ate Beige aes ogo ae poouryeq axe ejep oy) LOYM S[opoU! UOSsiog UT seLTEAGD e SuETUIO Jo SIAL 
SNAG ‘Vv sourer puke NASUALLAd “YUE 

1B Gy Pare a ak Sa Oe As aa is 0h ERE AO, LP $1S9} [etuOUTg pue AouITY AA -UURY 94) 10} AJOay) syuTod-jno Wopues Jo suoneor[dwy 
SAGUVAMGA ‘eP “ PeyoTW, 

Lite (REESE OSS as J0}09A WOpurI JaTYOUIG ke JO sjuoUOdWIOD 9y} JO SUOTBUIQUIOD IesUT] JO UONNQM)sSIp oy) UO 
ONOSHD OH-8uno x puke LSOAOUd ‘A 2330 

GGG ene betas ten te eee eae eee eet eats wun op UO s\UIENsUOD YAM LORNAUNSIp oneUNUts K}eouDUds 
NOSSWNO NP] pue YAININCHNOA enbruru0g 

PSN eR ES TCR OIRO DO ORS ORRG ORG 1G £ Ayisuop Surxmw ay} JO UoNeUNso Jewmndo pue sjapour oinyxrw FeuoNsoIq 
OOM 3u0X-ef pue WI WL 1910 

FOR AION BIOS hPa tS NE ernie cstae Meme nth A eS OF sis}sn[9 Jo Joquinu om SupeuNSA 
NVWIVUd Opseory pue OYA [enue ‘SyATNO omojuy 

fol ok of ak a INE ERE RE EASA TEM GT NO RPE UOU9}LD UOTRULIOJUI sakeg oy) Suisn syutod asueyo o1e1 prezey SutAynuopt 


:Aouanbalj a1J ISdIOJ JO AJO}STY OY) BUNONSUOIOY 
Cdae “f Wein 


€vE « siaje" ee" ele, 0 elle. alg ois, © oNojece tells 10 (ers) sis bois) eis nels) e /ellel (el (e,'6 16 )\0l s'0) Sie .6 00 suonnqmuisip Joud 0} J0}ORJ sokeg Jeuonoely ou} jo AYANISUIS 


NVOVH.O Auowjuy pure INVITOINOO Bune) 

A Faia ca a EO eG OS a OG OG OS SOANeUIOIE OLoureJeduou JsuTeBe IJ JO SsouUpOOS Jo JUDUISsasse ULISaKEg 
NVDVH.O Auoyjuy puke OULSVO URAT ‘f ‘INVITDINOD Bune) 

I I€ Bile) 6:56) Sle yale lo leleliele\ ele wtere\le)(o cols’ (e,.60) 6 (a).816) iiale sfiel ens slelie.dlelatals ele iv.) 0.618) (6 elece sjepoul onseyoois ul souln suniem jo uonoipeid uvisokeg 
UVZVAUNZNH ‘A ewedy pue YA TLA ‘MA Prewoy 

LOE 509.16" 'p eel O50) 91 \0.'@ 19) 6.8) 8) je He © 6100 8. 0) .¢ “Sid aie) 6 @ io..6 le eleeiens @8eLeKe) edie) el lellehellele wires: e1el o.eMeven arene eyep proyseiyy JOJ UOISSOISOI oujourereduonN, 
YaTIONW /N PINsiy 

os Mate ae ae SE ie SOE juopuadap oie JOLIO puke SIOSsaIZ01 UDYM UONCUINSS UOTSsoI3oI do}s-om) DUJoWeIedUON 
ASANId suo 

Te [OPO YAS Yor Jo UoNeUINSS Isnqoy 
ANSAHON eel, pue NVACGOTIA ULeEW 

7p RS aOR I: Cea ECs Fah PES eH ROR EI I RIT cs uoNesTep JatpNo sareu Zulssosse :SONSHeIS SO} puke syenprsal UOISsaIZ9y 
NOLTINVH (DO PlAeq pue SINdnd ‘f 9!999d 

[HZ oR sin ake elBiclioncPear tats ale Rae iage Pes: rai als eB lebave selene agen eva irons steal md sanjeuod poseq-[apoul yyIM uOIsseI3a1 pozijeusg 
AVSWV4 ‘O sows pue NVINMOGH “d Aouen 

CoC ®: Sis) Sc eewy eh lene '@ 0) ee) 6:91 9.0 1016\;9 Led, © Wiie,..0):4 10.1, 6/16 ¢ 0,16 \e) SNe aielie eo lebeslofeelisie 60. e) Leis 16) suonouny Teonsneis Joj s|Opoul uonenbo [enuslayjiq 


AVSWVA 'O somes 


Shb-S7Z “0007 UINfaune ‘7 “ON ‘gz QUINOA 
SHYALLVW SAC ATaVL SLN@LNOO 


anbysyejzs ap auusipeuesy anaay ey SIIJSILS JO [VUINOL ULIPEULD 94], 


ie 


he 11h: i 
a « 
(ws 


aye 


pee e in yee i, 
et 


y Li eee | Pipyytye 


) is ere 
ON ei ae 
an Ctnneti 


C-2iirow 


v 
enon nae 
dromyehs Le bt hs § t 
r 
aA 
Ay 


wpa ir AiR adn: . 
AMA's atv ¥ ., ) ak 
NLP 00s 


tn ie oe Vad 
| :. 


ALY 
A ad eh nse fe 
Aas pene ” a ad i 


AAA Cl Rs guy 


18: .} 


ae 
ek wai 
i pene + 7 "saa vA + 4s ny 


he 8 ae Keanys 


irs 


F 
ne 
sf | 
2! ui 
ia 
* 
~ 
a4 4 i> 0 
. 
is 
r i . 
i 
if 
o 
j < 
‘ 
6 
he 
’ 
he ‘9 
* 
1 
AL 
f 
gin 
i 
‘ 
Wy Ps Jj 
a ale 
mf 


a 
’ H Tha ea pan 
f : e 
Kab). Syne ee 
: y = _ 
; jh TOR, 
Cor 
‘ = [ f ‘ | 
ss Loe . : 
= | a 7 
} ~ as 
4 
re Yi 
: A oe) 
: ¢ RDS 28 Getter tie! pore eat 
ary vot Tt 


ny TRE tikes Do onmuycmtayS : 


Pits ae ar Le areiee fe 
= 
\ 
. 
if 
“Lew Mf 
* 7 Owe’ § , 
1 
cA ¥ ba 
oe 
‘ig 
wd pn 
een ar eae 
an a 


} ; 1 
esye 
? 
ea lisesee ery Tat 
* i i A 
. be ‘ ‘ds 
a ae 
= * t; | i 
a a) | te bk al 
y ¥ : 
. Wwe. 
= . , ‘ » . 
“t 
, ” 
. t 
% , ¥ if 
ae ay, SP ee a te — 
© mip Sy 
poor eriniey ev <7 ey, we 
aah il al - . woe Ses en 
; i] > 
Os a : 


a ea: Ny a i, ra Angee fs tip, = - 
F, i , aia _ Sieger) ei a Pkee 


r f : p. . ae e hee *& ou xe 
Cai een tpl oe ae 


- 


‘uotoepal P| 39 aIydesBoAjONp P| ‘UONeUIPIOOD v] B Wodde 


Ing] Inod ‘sa8eusul sap saidne saygnbus sap sapoylsul sap UOISIAIG e] op AVY “DH 19 aI] “qq ‘aIqeuie’y] “=D ‘JOM “+ ‘[IeqIoD ‘D ke 
JOULSSILUUOIAI OIJOU JOUILIGKS OIISEP UO JUSUIOTCUL] “(UONONPLI 19 Sa[[IIdIJJO SoNSuL] Sop UOTSTAIC) I[NeaLIIg “] 19 (UOISNIJIp e] 
dP UOISIAI) a[slasnesg ‘f :QQOZ nod onaal ev] op soJguINU sop UOTNONpoOd e] B anqLnUOS jUO Nb xnI99d JUSUIOTeS9 SIDIOWIII UC 


Ajisdaaluy) pADasDOH] ‘KYSAC|SCZ, “VW 
piupajdsuuad fo Ayisaaaiuy) ‘OWNUeZ “|Z x 
ppoun, anbysijojg ‘SUNK “MM x 
oop4aqoy fo Missaaiuy ‘NAA ‘Dd 
dgJuag Yyoavasay uoiuidg: JDUOIION ‘INOM ‘WY 
ppouvny anbiysijojs Ipusn “We 
‘QUT ‘JDJSAM, ‘BIOQSYCAA ‘[ 
‘OUT ‘JoIsay “WeI[eA “A * 
SONSIDIS 40QVT fo nvaing “SQ ‘ax ‘O 
ppoun, anbysiynjg ‘Ke|quio1y, ‘S$ 
uoljnULAofu], | 
ap asdjoupy,] ap ja anbtsijojg 0] ap ajDuouvN 21097 ‘ILA x 
ppouny anbiysijnjg ‘a312Q9uUL, “WV 
ppouny anbiysinjs ‘JaxMS ‘qd x 
Ayisdaaiuy) 210j¢ o014C ‘Kuseys “q 
INAAIG SINSYDIS JOANJJNAASP JDUOION “UWS “LY 
uojdupyjnog fo Ayisaaaiuy) ‘IQUUTAS “f[°D 
AJISAIAIUL] AISDAY UOUNY “IAG “Y 
ppouvy anbisijnj¢ ‘ysuls ‘dq’ W 
uosIpDY - uisuors1y fo Ayisaaaluy) ‘OeUS ‘f 
AJISAAAIUL] AAAISIY UAIJSAY ASDD “YSUCIPSS ‘f£ 
puvjyonp fo dsaaaiuy ‘Woog “W 
aINJIJSUT UDGAT) AY], ‘U2INIYIS “fy 
Sajasup soT - p1usofijpy fo Aisaaauy) ‘IaXUZYOS *N 
Saigojouysa], vIpsONa] ‘apues ‘| 
‘DUI JOISay “SNY “Y 
[DADT yiSdaaluy) “\SOATY “d-"T 
AJISAIAIUL) UOJAJADD ‘O’Y “A'N'L 
ppoun, anbiysijojg “unoouey “q 
ppounD anbysijnjg¢ ‘a{[tAouuand *g 
pjdaqip fo Ausaaatuy) ‘peseid ‘N'D'N 
Jaqang np anbysioj1g 0] ap injusuy, T ‘aAuUeld ‘N 
Apisdaaluy) MadgaH] ‘UUPULIDSJ2q “qd 
Arojouysay, foainjysuy syjasnyovsspyy ‘24YOStg *S-'f 
ppouvyg anbijsijojg ‘SION ‘qd 
Jans] ‘sousyois fo ndaang [D4Jua_D “UeueN *D 


* 


te He 


ppouny anbys1jnjg ‘Kauojoy ‘f£ 
ppouvn, anbiysyojg ‘stInoyxIoW “d 
ppouny anbiysinjg ‘jae ‘Hx 
ANSABAIU) IDIG DUOZIAY ‘IYOT'S x 
SI1jSIjojG fo nvaing uDIjDAISNY ‘d1IDeUTT *S 
ppouny anbisijnjg ‘aa{[ear’] ‘d 
ppouny anbys1jnjg ‘ayonoywy] ‘We x 
ppoun, anbysijnjg¢ ‘joey ‘N 
ujooulT - DYSDAGAN Jo Ausaaaquy) “LIYeT “q 
snsuay ay] fo nvaang ‘Sf: ‘Jourery ‘Wo 
ppoun, anbysijnjg ‘QIAdQeAOY “Wx 
IMAAIY’ SINSIDIS JOANJJNIAABY JOUOION NOY “S'd x 
‘OUT IDISAaf “WITY *f 
SINSIDIG UOYWINP sof dajuay jouoYyoON ‘uewyney ‘S$ 
‘OUT JDISAaLY ‘“UOYLY'D x 
‘OU ‘Wwjsay ‘surypne ‘qd x 
DIDUIG DIWAapDIP “BueMH ‘S-'f 
YN ‘sousyvig ouoywy 40f 2140 “MOH ‘dd 
ppoun, anbiysijnjg ‘nojsolpiy “WW x 
ppouv anbiysijnj¢g ‘ourquiey ‘f 
AISAIAIUL) BIDIS DMO] ‘I2[NAV'N x 
SINSIIDIS A0QVT fo nvaang ‘S‘/: ‘e8unTA “TL x 
ppoun) anbysuvjg ‘A910 ‘d « 
sanbiuouosa 
sapnja sap ja anbysijojs vj ap pouorjou jnjyjsuy ‘IAA ‘O-'f x 
SINSIJDIG UuowoNpT Aof 4ajuay JOUOON “UdYOD ‘WH 
snsuay ays fo nvaing ‘§') AID “AZO 
oopsaoy fo Ajisaaauy) “UsyD ‘f£ 
ppouny anbysiojg ‘Iowep -D'Y 
snsuag ay fo nvaing ‘S72 ‘[]aMwed ‘fd 
‘Duy jojsayy “1O\UeD “q 
ppouvy anbysiyojg ‘Jopulg “V'd x 
aINjiJsUu] ajsunid4] yIAvasay ‘INUWUIIg ‘d 
uojdunynos fo Ajisaaaiuy) ‘1a819g *X 
OLADJUGC) Udasayy fO ApiSdaaiuy) ‘BSnoUT[Ieg “Wd x 
snsuag ay fo nvaang ‘$') ‘TleA°M 
ppoun,y anbysijoj¢ “uoulnesg *4-"f 


‘stoy oun .p snjd adionsed ev suuosiod ej anb anbrput onbstigyse uA ‘OO0T 
o9uue | JURINp spore uN .p onbnt9 ey airey ap 9}da00e yuo Inb ‘sayueatns souuOsiIad sa] JOIOJOUISI asap ajanbua, p sanbiuysa 


SINAN GD YAN 


Ste 


“COb-Str ‘8 ‘SousyDIS jo10[E fo jousnor ‘suonesedo KoAins 
Joy suoneordunt say} pue spoyjoul DIS WO JO MOIAI Y :UONONTIO9 
uoneuloyur AdAIns paysisse-loynduiop (7661) AW ‘SNAAM 


“yS-Sh ‘SQ ‘uollpIOOSSp 
[D9NSIIDIS UDIIdaUp ay] {Oo [DUANOs *SMITAIOVUI ADAINS QOPJ-0}-99RJ 


UI so]qnoy feuonoeJajUy ‘(0661) “A ‘NVGUOL 1° “7T ‘NVIWHONS 
‘ZO9-9LS ‘19 ‘Ap4ajaonG 
uoiuidgC 21q9nJ (10a JUSUIOINSvaUI AJAINS JONPol SUIMIIAIOIUL 


[euonesiaau0d so0q (1661) ‘D'd ‘CAVUYNOO 19 “AW ‘YaAOHOS 


WINNZ ‘Wroyuueyy 


‘OZ/16 §=WOUAQsieqIy-VWINNZ ‘eseIyUI, JoUTS asstuqosiq 
‘puelyosjnoqg  yyqndolsopung Jop ur osulayskssSunseIog 
J9}ZjN\s9319)ndul09 Zyesulq (1661) W ‘dISNHOS 


“ullIOg ‘EOI ‘AIC Sep csordedsuoissnysiq 
‘Joue_ syOsMUOU0D20-01Z0S sep Inj aqoIdyonsssunzuesi1q 
Joule usIsog pun uonyuny (3661) dHOS AddNYOLAa[OUd 


‘910N-96 
ON UdOHV ‘SedIAJOS ueUINHY pue WTeOH Jo yuowedsg “s"f 
“AQAING Pjoysasnoy] aU} Ur JoIAeYyog JWapuodsay pue JOMaIAIO}U] 
Jo siskfeuy (9661) 'S ‘LXITd ? “O “TIANNVO “I ‘OUAANASMO 


“VSN “oqiy uUy *191U9D, 
Yoreasoy Adaing ‘jeu yoddey ‘smaiauajuy Aaaung ainpipuadxy 
[DoIpay= jouoyoN ay fo janpuoD pun usisagq ay) uo YyoAvasay 
(C661) ‘D “TIANNVO ?9 “S ‘LXITd “L ‘Ada “1 ‘OY aNASAO 


"€E9L-8SL ‘UOlDIDOSSP JDI11S1jDIS uDIIAaUIPp ‘Spoysayy 
yoivasay Aaaing uo uoijrag ays fo s8u1paad0dg ‘MIA IU0}S1Y 
pue Jemjdosuo0d & :spoyjJoUl SUIMOTAIOIUL Pojstsse-JojndUI0d Jo 
douridaooe ur s10198-4 (9661) “A ‘MANGA 9P 19 “TM “STIOHOIN 


‘066-986 ‘MONDID0SSP JDIIISIJDIS UDIIdaUp 
‘spoyjayy Yyodvasay Aaaangy uo uojsag ays fo ssuipaa201g 
{SASAINS JUSUIYST[QeISO UT UOTIET[OO BIJep JO} UZIsap suonsonb 
pednois Jo Sunst] pus ssoneq st yoIYA (9661) ‘Hl ‘YAAOW 


‘OW ‘SINOT Ig “YOdVV 2) Jo Sunso 
jenuuy,| @ g1Uasgid sJoNIy ‘souIojINO MOIAIOWUT UO s}aTTq 
uBIsoq esreuUOHseNd (48661) “TH ‘UYAAOW 3° “Of ‘ANOOW 


“snsudc oY} JO neaIng ‘S'~ ‘YoIRasay spoyyopy AOAING JOJ 19}Ua_D 
‘yodoy jeuly — juowUedxg plely poseg-o1do,sposeg-uosiog 
LLVO/SOV “(®8661) "TH ‘“YHAOW “OL ‘AYOON 


XL ‘omojuy ues ‘u0TdaT[0D UoNeUIOJUT AsAIng 
Paisissy-Jondwi0D uo sduUaIaUO_ JeuoNeUIOU] ‘96, DIS VDI] 
RI & gyuesgid o[oNIYy “sjuounNsuy AsAINS ployssnoy poaisissy 
-Joyndwio_Z Jo} usIsoq poseq-s1doy ‘sa -uosiog (9661) ‘Of ‘ANYOOWN 


"9ET-O7T ‘€9 ‘AjaajaonG uoiuidC 
INGnd “JOMUS IBVJOAOD ASAINS JO DINOS & se S]UDUIDdISESIP 


ployasnoy-ulypIAA {2194 Seat] OY SMOUY OYAA “(6661) “A ‘NLLUVIN 


"XOSS ‘[[-L6 ‘odueYy_ [elo0g-o10T UO aUAD YoIeasoy 
OUSA «1 OP [freaky ep juowumnsoq ‘Apmg joueg eurpnysuoT 
® Ul [qVO 0} SuMAAUOD (L661) ‘N ‘NOOW 1° “H ‘aranvT 


‘OCP-IZp “ACTEM IOX MON “ABojopoyjayy Aaaing auoydaja 
“(spq “Sroqgsye AA “f 19 “STIOUSIN “TAA “Aosseyy “Lf “819947 “AT 
‘TOUII ‘d’d ‘S2AOID “'Y) ‘Spoujow! pue seatyjsefqo usisap :e9 
JO} UsIsop alfeuUoNseNd (8861) “TA ‘STIOHOIN #9 “9'D ‘ASNOH 


“617-607 ‘I ‘sausums jo1f[o fo joUAnor ‘BUIMeTAIO\UI QUOYdeTa} 
paisisse Jojnduroo YIM USIsop oIteUUONSENd ($861) ‘D'D “ASNOH 


‘OW ‘sInoT ‘1S ‘AOdVV 2) JO Suns0] 
Jenuuy .[ @ Juesgid spony “JUOUINSU] STHN 94) JO Uonenfeaq 
Aiiges(y (8661) (W ‘SHONA 29 “dW “YAdNOD “d's ‘NASNVH 


OVI 8|nNpow un suep suolsanb sep aipso,| je sues0e Sep uoldeou09 ey :syon ve? 


"69€-9SE ‘8p “Mp4ajavnGo uoiuidy a11qng 
“sjuopuodsal puke SIOMAIAIOIUL UO JO9JJo :BUIMIIAIOJUL QUOYdeTO} 


paisisse JoinduloD ‘(/861) ‘W'N ‘ZLAMOIHLLVW 1 “Wa ‘SHAOUD 


“(ssoid snos) sousynyg j0191f{0 
fo jousénor {JOBUO] SYR) SMITAIOWL [qWd JO [dW Od :si09IJ9 
ABojouys2l, (0007) ‘S ‘NASNWH 3° “W “YddNOO “W ‘SHONA 


‘(anrered ) AdTI MA ‘“Malasajuy Maaung ayj ul 
aINIDA PUD UOlJIvABJUT ‘adpajmouy j1ov J, puv uoyozipavpuvjs 
‘(USMNOZ JOP ueA “"H ‘Jayjowyos “DN ‘elysudaj}S-dooynoy 
‘H «‘preudeFWeA ‘dd ‘Pa) $ ‘SMOIAIOUI §paysisse-Joyndui09 
UI UOT}ORINIUT UO AZoJouyoa} jo yoeduit ou, “(100Z) ‘W ‘SHONA 


"667-P82 ‘bT ‘atsojoizog anf Yiayosnaz 
“‘SUNYSSIOJOseIJWI) ~JOp UWId[goIg Jjne uUsOMIUY IdIUTY 
“‘BuNTeIJIG DYOSTUOJo[9} 91ZIMSIBIJOINdUIOD 9Iq *(S661) ‘W ‘SHONA 

‘UOIUL) SBRTIOA SISOTOYOAS :WIOYUIO MA, 
‘SuNseIog YOsSIUOJaTO} 93ZIMsSeBJoIndwWIOD sIp ul SuNIYyNUIq 
‘dajnduio pun uofajay, jiu sunyrsiofasvafu) (7661) W ‘SHONA 


“UeSIYOI 

jo AytsyoatuyQ, ‘yeuy woddey “Asaing MotAsoyuy Aployend 

(qO) snipusdxg Joumnsuod oy) Jo} usiIsaq juoWINNsu] [dV 
‘(L661)'d ‘SWUVdS 19 “A'S ‘“NASNVH “W ‘SHONA “d'W ‘WAdNOO 

“PSE-6bE 

‘UONDIDOSSP JDI'NSYDIS uDII4aUp ‘spoyjay YoAvasay Aaaungy 

uo uo1jrag ay} fo ssu1paaz0d4g ‘[qYD Ul sonsst [edIso[opouiop 
(6861) ‘DO ‘AUVSOM “WU ‘SHAOND “dW ‘WedNOO 

"YS-BE ‘ZI ‘maiaay sainduoy 

a0ua1gg 101909 ‘([qWa) BUIMIIAIOJUI [eUOSIOd pojsisse-Joyndwi09 
PIEMO} SOPNNL JOMIIAIONU] (661) “‘D “LUNA “dW ‘WAdNOO 

“KOTEAN 

SYJOX MON ‘UoIIajj0D uo1jonusofuy Aaaing pajsissp dajnduoD 

(8661) (SPA) “f ‘A TTIAY.O 9 “TMA “STIOHOIN “‘f ‘NILLUYVW 
“TZ “WAV “f ‘WAHATHLAG “da “WAV “dW “YAdNOD 

“LSI-SbI ‘OL ‘Mataay saindwoD 

anuaiag 101206 ‘([qWD) BUIMITAJO\UI Teuosiod paysisse-Jojnduio9 
syoreases AdAIns ul AsZojouyse} MON (7661) ‘dU ‘UAAVA 


aIHdVaOOITala 


‘gsodxo Juasgid np saindti9}Ue SUOISIOA ap jolns 
he san suonsadsns sap twINOJ JUO sowAUOUR SINdJeUIUIeXS 
xnap jo Boop] Aaljjor ‘youwe’y] powyseirg ‘Iadnog yoy 
‘gousLigdxe ajuasgid ey e odiored yuo tnb sInaMmatAJojUI S39] 
SUOIDIOUIOI SNON “666 LW QT 9] YOIRssoy uoruIdO sIIqng 
JO} UONRIDOSSY ULILIOWY,[ op a[enuue sg[quiasse pC 
FB] ap UoIsess0,| & 19 ‘“Q6GT FEW IZ a] UeSIYITW 
NP SUSIOAIUL] | BP YOIeIsoYy [eIoog JO} oyNjIWSUT | B NUD} 
enbojjoo un e agjuasgid 919 & syeI[NS91 sap oted ou 


SLNGNGONAaNAY 


“UILLIO} JJ INS STeSsa Pp 19 SIIOJeIOQL] Bp SdUaLIadx~a ,p 
uosIeuIqUiOD 3uNn.p opley ke uonesTyN,p gyyloey 


N 


ee 


RI Joipnyg OUOp WeIAOp uC ‘asuodel-uonsonb snssado1d 
Np Issne 30 JUBPUOdEI-INIMOIIAIOIUL UONICIOIUI J[QuILIQA 
R] Op UONUOJ US sUONNIOS sou ja sodAjoj}01d sou stewel 
SUOIOJUOIJUOD 9U SNOU ‘UTe1IA} 9] INS stessa sues ‘ueLINOd 
¥q ‘seo op sIquioU puvIs suIOU UN yUdsIxe “JUoNbgsuOd 
Jed ‘ya sjuediomsed sop jofns ne saieiep snjd sjuow 
-OUBIOSUdI SOP JUISSIUINOJ ‘9[Q.UOD snjd 9}x9}U09 UN sup 
JUOJ 9S dILOeOQGR] Op soousLI9dxe sa] “INS UI “suIOSAq 
sou sno} & sed yuopuodel ou am0yeloger] op seoustadxo 
So] ‘MOIAIOJUL.| OP dgINP ek, Ins onb ouIQUI Op INOMOIA 
-I9JUI | OP JUSWISLOMWIOD JT INS sJaJJo SS 39 MOIAIOJUL | 
OP s[eIN0S UOISUSWIP ev] JosATeUe SUO[NOA snou onbsing 
“UILLI9} B] INS S2OUDLIdXS SoINe ,_P UIOS9Q JUSWIICIA SUOAR 
Snou ‘snssap-Io ssguUOnUSU suOT}sonb s9y JonTeAg ,p UIFY 
“UOTNRSI[IN .P S[IOvJ Op Sapnjg sone .p IOUS SUOADP 
SNOU ‘INIMOIAIOIUL,| op Wed e] op spOJjo sop jo JUNIE | 
op ‘sdwia) np Joussed9 e Jopre yweunod aja yUOWUIOD 
y@ saguuop sop oyyenb e], e olmmu yeunod OY] sueId9 
sop setjnonsed uondsouos oun sinsow o[jenb suep Jour 
-Id}9p op UlfY “Ued oNe.p seLIEqUID sUeS 30 JUSUWOAQTIG 
‘WudUIZONOp e[NoJep as Inb MoIAJO\UI oUN 30 “ued oUN.p 
J QeIJ 19 BIQUIeA oINSoU op snssoodoJd uN oUS sTUOIdWIO0D 
SINIJIOW sot suleNUS s139}eNs ajonb sed sinofno} 
SUOARS OU SNON ‘soguUOp sap sWgUI gyITenb ey] e oIINU 
ap onbsti [I ‘ainsow ap snsseooid np sfeloos uOTsuSWIp 
eI oqinyied OY] aleuuonsonb un Ig ‘suonsanb say ayua 
jusuUdTAINs Inb syusWayIOdWI0d sap 39 JUBPUOdaI-INIMOIA 
-I9JUI UOTDLIOJUL | OP B[VINOS UOISUSUIP v] ap a}duI09 ITD} 
op ueLOdUII JUSUTITeSE 3S9 [I ‘SIAR UOUI B ‘SIOJaINO], “sed 
onbeys suep ndu0d uaiq y0s asuodgi-uonsanb snssad01d 
a] aonb 3a InoyosoYyO Np sulosaq xne spuodgs OY] leu 
-uonsonb un nb jueyoduit sa [I ‘npusjus ustg ‘syuepuodg 
J9 SINOMOIAIOUI ‘sINdYydIOYO :s]UsI[D syusJaJjIp snod 
souryodut | op ie Inayeurpso Jed sgysisse sorreuuonsonb op 
uondeouos ej aonb sasoddns e uonsenb ap a1uad 99 JonTeag & 
jURAIIS sgI[NoIWed s139}e.Ns oun Josodoid sresowe ,f GuoUT 
-d][QUUOSIOg ‘seessnod snjd saydsayoel op opie | e asuodeal 
sues $99}So1 suonsanb sap Japsoge .p nol] JUsWIETeS9 v A [J 
"$}e][NSoI Sap Jojuasgid Inod 19} don yuepusdas 
1S9 [] LO139Ie.INS J][9} OUN Pp SO}TWUTT So] JUOS Satfonb 39 SMTA 
-IQVUL SOP SWORITJJO | JOIOIOWL ,.P B]INN [I-}S9 SIoURISUODITD 
soyjonb sueq {ouO} Pe] Ins saxe suRID sop & NO sols 
SOP B SINOIAI SEJULALP SUOAR SNou Is |I-}-ossed 9s ond 
:19-2][99 1S o[eJgUEs UONsenb ey] *(‘qJa ‘sajueAINS suOTsonb 
‘sajuapaogid suonsanb) areuuonsenb un .p uorNndexe,| suep 
INOMOIAIOVUL | & SIUINOJ B[[9NIXI}UOS UOTVULIOJUI ,_p SgIZap 
SJUIIIJJIP JUSLOdWOD sousLI9dxe ajUasgud eB] SULP sdgIPNId 
SUOISIOA SFT “SOPNINE Pp SOLIOS9}e9 Sap Inod ya aseugU UN.p 
SOIQUIOU SOP dISI] oUN.p Infao onb one 3}x9}U0D UN suep 
SOTJONIOVJ SUONPWIOJUI PP UOTOUOJ UD Sa[QpoUl s}UdIOJJIP 
Joredwiod ap ulje UTeLID} oJ INS sTessa sap JUSWOT[ONIOe 
SUOUSUI SNON ‘alleuuoNsonb un.p syUusWsas sone p ke 
aBeUDU UN Pp SdIQUIOUI Sap 9}SI] oUN Pp JosodsuLy JUSSSIE] 9S 
$]e][NSOI SOU IS JOUTULIAZ}9p op nat] e A [I ‘sntd aq ‘apoyjsoUl 
auuog ev] sed jos ou 90 ‘sasuodgl sop opnqoexo.[ op 
ana op jurod np no (neapses) [e190s Jo JITUSOD ona ap juIod 


0002 euquiasep ‘ajenbua,p senbiuyoe | 


un,p anb stew ‘UOIsI9A 91199 & Joydepe s JudWIOTIOR] JUsssind 
s}uepuodgi sa] anb ynad as |] ‘sjuepuodgi say sed sgsuouIsUI 
1SO UONPULIOJUL | JUOP UOSeJ LT B pUOdsaLIOD UOISIAA 3})99 IS 
sed suoaes ou snou ‘snid oq *3jUSI9JJIP UOTSIOA BT op yofns 
ne uotutdo Ingj Jonjeag,p Jueyodut suop 3s9 [J ‘suonsanb 
SOP SIPJO 199 ap pies] & SJUBpUoda sap UOHORIsHes eI [RU 
SUOSSTBUUOD SNOU ‘dUOD Jeg “SUIQY} UN INS daXe UOISIOA 
RB] JUoIQJ9Id SINIMOTAIOIUI So] onb gnbipul v (qR66I ‘2R661 
JahOW 19 BIOOW 9661) SOO] Jog “WUouLs[IOey Joydepe,s 
yusanad juepuodsl 9] 19 INOMOTAIOJUI | JanbNe MITAIOIUT | 
9p Jolnored USWs[NOJOp UN JI1DSNS dUIQY}I/9][3 UOISIOA 
ze] onb juonbipur saguuop sou :juepuodgl-inemotAsoqul 
UONIVIDJUI,| Op ona op yurod np Inajeurpso sed gistsse 
eleuuonsonb un JdJOI[9We.p Ulfe oITey yney [I.nb a9 
*(sasnaro 
-uepus} suorsonb a[durexe sed) saguuop sop syjenb ey e 
sed OSINU OU INOMIIAIOJUL | Op JUoWayIOdUION a] anb nainod 
‘soqjomjovy suonsonb ap souasgid ua ‘alqeiejoid owrgui 
no g[qeidasor ang weunod ejaD ‘uoNnenyis ke] op UOTOUOJ 
Ud IIBV9I OP 39 UOTSIOAUOD LI ap Sa[sei sao Ja\OadsaI 
OP INOMITAIOIUL,| & JUSWoULIOd SWIQU} UN INS spud} suONsoNnb 
Sop dIpPJO UN 39 OILS ouN Ins sgpuUoJ suURIDa sap UOT) 
-da0u0d oup “sarqisuasut oered sed ou op ule uepuodar 
9] Ins osvjuRAep oxe JUSUTALIOdWIOD UN dUOp jUa}dope sI] 
‘ated ua nb uonsenb ey] Juasod au no uonsanb ey Jasod sues 
SOUIQUI-XND BSUOdAI B] JUDANOS JUDALIOSUI SINOMOIAIOJUL SI] 
‘gsuodal be] & UONUa}e JoyIod sed ou op uoIssalduut | IoUuUOp 
sed ou op ulje ‘(S661 ‘Z66I) [109 19 BIOqUasSYO ZUSI[NOS 
UO | SWUWIOD ‘soirejusugtddns sjusWIaUsIasUaI Sop WUINOJ 
stew ‘agpuewisp uonsenb ex] e sed oyu] os ou yuepuodgl 
a] onbsioy jusurgstogid snjd asoduit.s ejag “juepuodes 
np ogt[nosed uoNens eI Jo}00dSal op no JUSWI9}eIPSWIUIT 
opgdeid inb 99 daAe JoUIeYoUS XNdIW & UOSeJ op UONENIIS 
PR] OP Say [epoU sa_ JURASP SINaMOIAIOJUI Sat Jed soyodde 
sjusuiajsn[e1 sap yUSANOS jWUOdJQ]Jol syUoWEsULYD sod 
onb guutyye UO (E661 ‘Z661) ‘1109 19 BIoquesyO “prepueis 
MOIAIOUI.p soinpgsoid sep juourenbely yUoUSIO[S,S 
SINIMIIAIOUT so] aonb yuanbipur sjuswayodui0s sap 
OBePOd Np siejNsal Sa] ‘UOTESIDAUOD PI Ap So[v19UES So[dQI 
Sop UOTDUOJ US JUOWIOLIOdUIOD INe] Ja}snfe1 op 19-xN9d 
JOULIO [I 1ed ‘pIeSd9 199 B SINIMOIAIOIUI SOP OY") L] SIN[IOVJ 
Q[Us oun Ins gpuoj sfepou un,nb gyUoUIsp 919 & I] 
‘IQIOIJQUE UD, Pp INIMITAIOIUL | Joydgdue sed 
WIADP 9U OY] oleuuonsanb un,p suvsog sap uondsou09 
R] ‘opueuwlep ud Inj aU UO.Nb sjuauIoUsIasUaI op snid WUINOJ 
juepuodg: un,nbsioy ‘stojainoy, ‘sefeqo[s suonsenb sap 
Jasod e uodvj op sanagid suonsanb soy JUSTJIPOUL SINOMOIA 
-Ia\ut say aonb sed suojeynos au snou jo ‘sayeqo[s suonsenb 
Sop Jasin e siInoyosoyo set Joseinoous,p sed ouop 
1138S ou [] ‘sduis) sugUI US oBeUSUI Np SeiquoU sinorIsnyd 
Inod sjuswiouStasual Sap WWINOJ 39 [eqo|s opoul un a}dope 
19-In[90 anbssoj Juepuodas 9] Jed stuINOJ syUSWIOUSIosUAI S| 
JISI[NN ,P INIMIIAIOIUL | B JURNOULIEd suLId9 sep UOTdeouUOd 
aun JOIN] d JUaSLIOAR] syeI[Ns9JI So] ‘algnbua ,p osreuuonsanb 
UN .p UoNdaouOd eI ap SIO] SaTeqo[s suoNsonb sap & SINOIEI 
JOAv .p sed apuewmosel ou INaINe | 39 ‘saTeqo]s suonsenb 


SOP B SINODEI JUO SINIMIIAIOJUI Say onb sed yuonbrpur,.u 
gsodxe juesgid a] suep sgfeusis syei[nsg1 soy ‘stozoynoy, 
‘safeqoys suonsenb sap sonbijdde xe yueystsuos snagid uou 
SJUDUISLOCUIOD Sap B SINODAI JUO SINEMOTAIOIUT Sa] onbsIO] 
seguuop sep siyjenb ex] ep uoNNuTUIp oun e qWeIpuane,s 
uo ‘ane | e suUOsIed ouNn.p safeqo[s suonsenb sap 
Inod s}ej[Ns9g1 $99 asT[eVI9UEZ UO J IS *SPIQUIOUEP SOUILIO sap 
g[qetogidde uoneUSWSne oUN JUSUTLUS UONRSTLUINOIA By] 
Ins somejuourg{ddns xneusis sop onb ‘aseugui-sioy ast] oun 
Ins sagpuoj suonsenb sap inod ‘amuoul UO (1661) “[[09 19 
uueullopury ‘snjd oq ‘suonsenb sayne.p asod uo J anbsioy 
asevugUI UN sSURp sagIquIOUsp soUUOsIed ap siquioU np 
a[qeiooidde juoulasstolooe uN g[ad¢ep & (666[) Ue ‘Isury 
‘QSVUSUI NP SoIQUIOUI Sa_ sno} nod sayeqo[3s suonsonb 
Sop B SINODEI JULAR UD PJEpUL}S MOIAIOIUI.p soUaNbes Fe] 
OP SUSIO[Y S$ INOMOTAIOJUL ,[ IS INOLNS USTUDAUODUT XNaLIEs 
un JajUasgidal jnod e990 ‘anbiZo;OpoUJeU JUSUIA}OLNS ONA op 
qutod un.q ‘plepurjs MIIAIOJUL | OP OUBIOTY,S INOMIIAIOIUL | 
sjonbso] suep seo ap niooe aIqUIOU UN 9319SNs SUEY) UN INS 
QPUOJ SuOT|sonb sap dIpPIO UN & Jd SATUS sap & sINOdaI FT 
“SMOIAIOIUI Soy INod ssnasejuRAe snd o109Ue 
1S9 SUOT]OUOJ XNap Sop UOsTeUTqUIOD UN Nb onbipul gsodxa 
quosgid a] ‘oUIQY} UN INs spud} sUOTJsoNb sap a1psO UN JUST 
-9[e39 JUL}dOpe Ud SMOTAIOJU! SOP 9WOVOTJJO | JOIOI[SUIe .p 
giqissod ysa jI.nb smuoW UO (GRG6I ‘e866I1) JOAOW 
39 AOOW (L661 ‘[[09 39 Jednod) sajjus sep yuesstuinoj 
INI] Ud SINIMIIAIOJUI SOP JUSIUApUAI J] JOIOI[OUIe ,p afqissod 
4so [I,nb juonbrpur ‘sanbyeursy) J2 say[iupenb sajapour sap 
INS UBSIYOIPL NP SIOAIUL] | Op JoJUND YoIeasoy ADAINS 
Np UOTesI[NN p [lovJ op sloOJeOQGe] 9] suep ‘soguoU 
suoAe snou anb saouatiadxea sa] ‘soguUOp sap o39aT][09 
Rl V1gTQOIe 39 JULPUOdSI-INIMATAIOIUL UONILIIIUL J STLOeJ 
Sd][I3 sap & SINODaI a] anb sin[oUOD op jua}ouLIEd gsodxa 
quosgid a] SUBP S}LIOSP s}eI[Nsel So] “INOMOIAIOJUI UN.p 
uonedionied vy d0Ae InayeUIpJO Jed ag\sIsse sogUUOp sop 
9199][09 ve] Inod ajgnbue,p someuuonsenb sap uondsou0s 
RI] INS UOISsNosIp e] B JUoWOddeI os sjyeI[NsoI SON 
‘uonsanb ev] ap ja[du09 93x90} np uonnedel 
sues ogtidoidde asuodgi oureysoid ey Itwinoy e yUepuodel 
2] Jople.p jo sjUoWOUsIOSUal So] SILIOSUI.p ‘UONENIIS 
RB] & JUOWoTIOVJ snjd Jojdepe.s op INdMOTAIOJUI | Y JouLIOd 
aigtnoed UOISISA 93199 sTeU ‘soULIOJIUN Jey B NO} sed 
UOS OU SyeI[NSeI say] ‘sdua} oUIQUI US soIquIOUT sInoIsnyd 
Inod SUIOUI Np NO sseUESUI NP SaIQUIOUL Sez SNO} Inod sjUSUI 
-QUBIOSUaI Sap JUaANOS syd jUINO} 39 ‘adeugUI Np souUOsiod 
so] Inod juowepider snjd sjuowlousiesuel sop wuinoj 
juepuodai oy sjenbsa] suep sed op snd outenue UOISIAA 3199 
(z {juawioonop snjd jue[NoJep as asuodgi-uoNsenb snssad01d 
9] 19 Uouaste snjd uoNsenb oureysoid e] Jodronue jueanod 
xnop say ‘asuodai-uonsonb snssosd01d np anbiso] ef e xnoruy 
juajdepe s juepuodgi 9] onb uarq Issne INSMTAIAIUT | “OUIQY) 
/a|{I3 UOTSIOA eT Inod ({ :MOIAIOIUI.] Op VgINP L[ op 3s9 
nb 90 Inod auIgY}/a][US UOISIOA BP] ap JUaWepUel INo[[IoW! 
Np SUOSIeI XNop JOUIO & Ope [I ‘s]qeIgprsuos sed js9,U 
yuepuodal np IMjao AMS Jo INOMOTAIOIUL,| op JUSWISOdUIOS 
g[ Ins sg[[upenb uondsouos eT ap yasja] Is owe 


OVI einpow un suep suoNsenb sep aipso,| 39 SUBD Sep UO!deDU09 BF :syony ZEZ 


‘yuepuodai np jusuay10du109 9] snod 
JO INOMOTAIOIUL | Op JUSWIa}ZOdWIOS 9] Inod ‘aginp ey] INs 
J9JJO,] & JUOWOATeIedWIOD ‘arqrey snd a109U9 39 suONsanb 
Sap ap10,, op mnfeo aonb enbsew surowl jso suviog sop 
BUOYS ap J2JJ9,] onb JouUONUSW op oyJOdUIT |] ‘sueIDS sep 
UONCINSIJUOD P| B UOIIAUD SIAN) UN 39 ‘SUOTJSaNb Sap SPIO ,| B 
sonquye og JUoANed UOTONpel 91199 op SJOT] XNOC] “UOITAUO 
%L| op suudskoul sginp vk] sJINpel op JouIod suey) UN INs 
gpuoj suonsenb sap aipso un 3 a3 ouN Ins sgpuo} suvIOE 
SOP PUIDYIS UN IOUIQUIOD ap Ie} YJ “SUOISIOA SoNNe SION 
sop one | no oun] Inod anb swey}/ay[us dA} op aTepoul 
UN,P dIped os] sUep ddvdIJJO snd UOdSeJ Op JUOUUOTIOUOJ 
sjuepuodsl sa] oUIWIOD 3NO} SINaMOIAIOWUI sat onb 
JUAIUOUI (SUOISIOA SaT $9}NO} Inod 9gsITWN JULIa sUOTsoNb 
SOP ZI[OQI] SUIQUI d[) OSLUDL UN ,P SQUID Sop 9)SI] oUN,p 
SUOISIOA oeNb ap uOsteIeduI0d a1j0U Op syeI[Nsel So] 


NOISNTONODO LO NOISSMOSIG ‘S$ 


‘sasuodal Sop SuIpI[eA eI Ins Joyo un 
Jajao9gp suoanod ou snou ‘sajqruodsip saguuop se] saide .p 
‘SlOjaIno], ‘ag}deJje JSa soguuOp sap suenb e] oinsow 
ayjenb suep JOUTULIAJ9p INVA UO] IS spuvIs snjd saguUOp 
ap safquiasua sap jueyodwios saguuop sop 9syyenb ey] 
Op plepur}s sinajeoIpul sone .p Jasin vipnej [J ‘soguuOp 
sop oyyenb ey] e sed yinu ou ald oun Ins gpuoj sepou! 9] 
onb uonesriput ayqrey oun nb juasstuinoj ou syeq]Nses sad 


aiqetogidde souaseyjIp sunony 
(s911039}e9 p) 
ynje1¢ 


(sa1s0891e9 61) 
SSA UI¥ .p sQUUY 


el oe cat a! €'l 


Gl eI Gul GA Cal 


suuosiad suuosiod suigy}+ sUWQU) 


ny + wuoUIsTA +9T1UH wows” + 1D 


a1qeue A 


UOISJOA BT UOTas oseUgUI Jed 
(gilgUgZOUIOY) Sa}DUTSIP SoL039}e9 op UsAOUI DIQUION 


¢ neaqeL, 


aqeiogidde 
oUdIOJJIP oUNONe s}soflueUI dU (AaseUSU UN,p oUUOSIOd 
onbeyo snod oun) saistoys sajounsip asuodal ap sati0s9ye9 
Op UdAOU! SIQUIOU 3] ‘SUOISIOA SO] S9}INO} INO “SUOISIOA 
SI] BJU I[QISIA DOUDIDJJIP DUNINe 9}SIX9,U [I “(JosULIIE,.p 
ann @ no pueway.p 720 e) onbipunf jnjye3s a] nod 
SUIUOS S9ALLIV Pp BQUUP | INOg ‘segUUOP Sap siIgUasOUIOY | 
op 3so inb 99 Inod souclgyjIp ounone suonbiewias 
ou snou ‘(¢ neafqe}) sa}oUNsIP osuoda1 ap sal0s9}e9 
op UsKOU SIqUIOU Z| JURUTWIeXE UY “(SotUINO} sasuodal sap 
QUIO}XO UOTJPI[BA SUNN SUOAR,U SNOU JUSWdsNeiNsy]eW) 
Jalnonied oseugul Ino] suep uoNeNiIs eB] op osneo ke 
SoJOUNSIP asuodal op SaLIOS9Ied Sop IISIOYO Np JUsTeINe SsII,s 
OUIQUI SSLUDUL NP SdIQUIOU Sa] Sno} Inod ssuodas oun IUwINOy 
jUO sjuepuodal saujNe So] SUIQUI IS UST NO ‘sed J] JUSWIS|[IEI 
189,09 Inb Inod xnoo yUoWIa[nos jUOs («aguUe sUIQUI 


L€?e 


PR] JUBpUad sNUdA SNO} SAUTWOS SNOU») Id9INODIeI Z| JUS}dope 
inb syuepuodai say Is Jonbiput yesasp ead ‘(sojounNsip 
esuodas ep sotiosgyed sion = gg6l Si 1 “L861 Ply 
eI ‘L861 esteuoyred 9 ‘¢g6] JUepuodas 9] :opALUe .p oguUR | 
Inod ajdwaxe sed) o8eusui np saiquiow soy sno} inod 
32 Jaljnonied yuoulsta un inod juepuodai 9] Jed sarstoyo 
SOJOUTISIP VsUOdaI ap SOLIOT9\e9 Op SIQUIOU 9] 9[NITed SUOAR 
snou ‘a[qissod JUSTUSAUODU! 399 Jon[eAg IOANOd ap ulyy 
“saguuop sep syyenb e] Jassieq alle} ap duop 39 asuodas 
Rl ap suouesowoy,, antoisoe.p onbsu («s9uue oWgUI 
R] JUepued ssALUe sno} SOUUIOS snou») sduia} suIgUI US 
aBeUDUI Np SoIquioU so] sno} Inod aipuodai ap ye; oT ‘snd 
aq ‘eseusuI np sjuenbgsqns soiquicul sop Inod uonsenb 
UIQUI BI & DIPUOdEI Op JUSUIOW Ne xNoINOBLI SUTOU JOS 
juepuodai 9] onb a[qissod 3sa [I 39 StoJ o[nas oun UONsonb P| 
ap Ja]duI09 9}x9} 9] pusjus yUepuOdsI 9] ‘sed sap UOIOdoId 
ou} oun sueq ‘jUepuodal 9] Jed soruINnoJ sasuodal sop 9i19U 
-930W0Y ,| INS JoZOMUIUI s WelINod uo ‘sIOJoINO, ‘UOTSIOA 
anne, no oun. suep uoysonb oun inod sayuenbueur 
sIngjea op uoTyodod ay10} snjd oun e sed suopuaye snou ou 
snou ‘a}ITeal Uq “sovdIJJo sed 3sa_,U SaguUOP sop syenb eI ap 
plepur}s Ino}edIpuI 399 ap asAyeue | onb a0s op ‘uonsonb 
oun Inod sajuenbueu sinejea sores saz) ap onb suoAjasqo ,u 
SNON “OSNaINOBL Jo BSUdIUT SUIOUI UOSe] Op asuOdgI PT SIOTe 
alepisuod juepuodel a] ‘suTuajepaid asuodei-uoysonb 
snssedoid np suoneoljipou ap asned e saguUOp sap 
guyenb e] op assteq oun ang Weunod juepuodal-InoMorAIOWUl 
sjuswisyoduiod sad ap asqissod juatugAuooUr up 
(quopeatnbe JuawWazyIodu09 un Juepuodai 9] Zayd sed oyfoOsns 
ou suoljsonb sap spo | steur ‘(7 nesfqe}) suUOsIOd/aT[LI3 
odA} ap ayapour un Inod surgUI JUSUSLOdUIOD NZ] JUSTJIPOU 
SINIMOIAIOJUL SI] ‘NA ‘SUOAR | SNOU SUIWIOD) “jUSUTBONOp 
snid Ja[NoJep oS op MPIAIOIUI,] e JouLIEd Inb 90 “uepuoda 
-INOMOIAIOUL =uorORIUI., spIoey Inb uodey oun.p 
quuosiad/at[Us odA} ap e[apowl a] suep suoNsenb sap arpso J 
@ 39 SUBIO Sop UOI}dsdU09 eT B DUOP JWOSSISEII SINOMOIIA 
-Ia}UI say onb usiq Issne sjuepuodal sa] ‘ajuarayjIp uodsey 
op juepuodsl 3] juel[INSIe ‘suei99 sap uoNdsoU09 ke] Op 
UOTJOUOJ Ud JUSWIDLTOGWIOD UOS JIJIPOW INIMITAIOUI | onb 
1So a[qIssod uoNeor[dxa anes eT] “sueIdg sop UOT}da0U09 PT 
op JULINOD Ne jNOo} Np sed juej9,U sjURepUOdal say ‘auoYdsTei 
Jed oguoul 939 B opnyd ‘J ‘ouUOsJod ke] Ins axe suonsanb sap 
dIpJO UN asin uO Nbsio] suei9g sap suol}daou0d xnap so] 
Inod suIQUI JUSIAIJIP syeyNsai saz onb afqenbrewsa so [J 


*sdua) ourgUl 
Ud IBLUDUL NP SaIQUIDU So] SNO} Inod sjusWIsUsTOsSUaI 
Sap WuINO}J yUepuodal 9] sjenbse] suep sed ap siquiou 
Q] JoUoWIsNe We} oWIQYy) UN Ins 9xe suoNsanb sap 
dIpsO uN ev sgjownf a][U3 ouN Ins sgpuoj UONdaduU09 ouN 
‘slojoinoy, ‘ajqeiogidde souasajjip oun sed oulenuo .u 
OUIQY} BJ Ins sxe suoysenb sap sIpJO uN B sINOdaI 9] 
(SQ‘O > @) a[qeiogidde uonses9jUI,p JaJyo UN yUepuadao 
JQAQI VSA[Cue 'T “OANVOIJIUSIS ITUDADP SIOJa]NO} sues 
“(ML‘9Z eNUOD %9O‘EE) sIQIeJ snjd jUaWATQeIDgIdde 
1Sd JUWIZIS,| INS axe B[QpOU Z| 19 J[[US eT Ins oxe 


000Z aiqweasap ‘ajanbusa.p sanbiuyoa 


J[QPOUI 9] aNUS VoUSIOJJIP eB] ‘oNUOD Jeg ‘(«enbiprinf 
NJe}S SUIQUI 9] SNO} SUOAR snoU» ‘«a9UUe dsUIQUI 
R] Juepuod snudA sno} souUIOS snou» aydutoxo sed) 
sduis}  owiguI =oud)6o8eusuI =o np ~—s souuosiod 
ap odnos un Inod no souuosied sa] sayno} snod syuouw 
-OUSIOSUAI SOP SIO} OUN SYIOUW Ne WUINOJ JUepuodal 9] 
sjenbs9] suep (100‘0 > d ‘a]jouuosied o189yens ey Inod 
MWL‘6| & WOWOAneIedUIOD %E‘Zp) sed ap uoTMIOdod 
sO} snd oun ay19sns onbyewsy} sjgpowi 9] onb onbipur 
sjuepuodg sap yuowayIO0duI09 np asAyeue oun ‘snidoq °Z 


‘(1000 > d ‘sapuoses g‘g) olteutpso souanbgs ev] yUua}90dsa1 
mnb sjyuoulg[g soy aonb (sapuoses Q‘p) sdwia} op surou 
juoWe[quiogidde juouuaid MATAIOUL | op onagid souenbgs vy 
Op QIJIp Inb INOMOTAIOVUI Pp JUoUUSTIOdUIOS UN Jed sgyono} 
SUIS] SO] :MOIAIOIUL | B Q1OeSUOD Sdur9} J] Issne soUSNTJUL 
RI9Q (‘suonsonb sop s1psio.| & UOMAUD %C/ 39 SUBIOD 
Sop UOIds9U09 FI & [RQO]S 19JJ9,| OP UOITAUD 9% CZ JonquNe 
nod uC ‘(UOTOeIOIUI,p afquiogidde yajJo unone ‘[Q0‘0 > d 
:suonsonb sop cpio ‘[Q0‘O > d :sueiog sap uondaouos) 
jUSUUTepuUsdepuUI [eqo]s 19JJ9,| JUDOUONTJUI sind}oej xnop 
sot onb onbipul soueLea ev] op asAyeue ou) ‘uosaid e nbsnf 
soguoul sopmj9 sop uednyid ey iInod sunou ve] ajuasgidal 
inb ({Z‘Q) auuosiadjusurs[a uoIsIoA eT onb (gp‘Q op Isa 
S9yono} sJUsUI[9 sop suUdAOW UOTIOdOId eI) a1Ua3 99 op 
sjUsWaLIOdUI09 ep snjd stoj xnop op snjd ouyeyUa MOTAIOJUI 
OUN & SWIQU}/|[US UOISIOA PV] Op UONRdI{dde | :sMaTAIOUI Sap 
enagid souanbes ve] & JUDWISATILIAI SJUDIOJJIP WO} syed sop 
JUDISOJIULL SUOISIOA oNeNb say] ‘yUoWsWOdWIOD ap o1u03 
99 Jed sgyono} seo sed syuouwg[9 sap uomodoid ey] gnoyeo 
SUOAR SNOU ‘plepUe}S MOIAIONUI., @ sed juepuodsa10s 
OU INAMOIAIOIUL PP SJUSUTS}LIOdWIOD Sap UOTDUOJ Ud ULIDE ,p 
Q[QPoul Np suorIsiaa aNenb say Joreduioo toanod op uy 


"10-xnao 
Inod sayeullogp sap 32 RI-xnao Inod sa8eyusoINOd sap ysI[4WN suOAR 
SNOU ‘INOMITAIOUL | Op JUUISLOdUIOD UTeyI9 UN Jed sgyono} (j) sed 
Jed sjusulg{e .p suusAou uorodoid e] 39 JuUepuodsl np yUsUIayOdUIOS 
ureyioo un Jed sgyono} seo op uorodoid ef Jonsunsip sp uyy 


100°0 > dase 


WT RE Sdwio} ouigui uo oseugu 
np souuossod say sajno} 
Inod uoneunojut | 
uno; yuepuodar a] 
sed 9] UOTES 

SQUBISUOD MOTAIOJUL | 
e sed juepuodsayi09 

OU INIMIIAIOIUT.P 
jusuayoduros un 

qed sgyono} sjusuI9|9 ,.P 
guusXoul uorodoig 


+xxbl6T %80l 606% Brrr 


*«%x9E'O 170 re0 a0) 8r'0 


guuosiod suuosiod ouigy}+ sulQu} 


ae + UOUIsTA + AIUD juowsiy + SHO 


UOISJOA EB] UOTAS JUepuodal np 
JUSWIWOdUIOD J9 INSMIIAIOIUL | 9p JUdWIaOdUIOD 


RIULAG 


‘Ina}euIpsO Jed 9\SISse a[npour 2] Jed soIsg\uUI ong sed 
yuvanod ou ya snagid uou yuepuodal np syusWayOdUI09 ap 


uonouoj us asuodgs-uonsanb snssaooid 9] Jastfeuuosied ze 
Uslq stew ‘sdula} np Jousred9 e sed ouop ajstsuoo ou gL0Ud 
R] “SNOU UO[Ig ‘plepue}s INBMOIAIOIUI.p sjUsWIayOdUIOD 
sop onb sduia) op surow ouop yuouueld sjuouoy10dui09 
SdD “UONPSISAUOD kB] Op sa[seI sa] Ja}oedse1 e 30 n{d 
jueyoseyo “Juepuodgs 9] Jed stUINOJ UONeULIOJUI,] JUBADP 
JUSISIJIPUI Joyso1 sed jNOA DU INIMOIAIOIUI J “AIUINOJ 919 
elgp e osuodgi oun ojjanbe] e uonsanb oun sasod sed ynda ou 
InaeMalAJoUt | onb onbipul vjao ‘stave aNOU Y “UONeSIAUOD 
eB Op so[sex xne ouoyuod snjd oipuel eB] op no uon 
-BSIDAUOD BT JOI[IOeJ Op 9]U9} ‘UOTsoNb vy ap 10exa 9X9} 9] 
sed I] OU INOMATAIOJUI | a[duroxe Jed ‘toinod9eI ap adedsa 
aun JUsJUESgIdoI sjUsWIa}IOdUIO0D sao op wedn{d e] :ayUeAINS 
UOISN[OUOD L] SUOIT) snoU ‘sJoI[NONIed s}e[NSgI S90 oq 


“SaTqe|eA SoINSoUL 
Sop JIU9}q0 & Opre JUSUS}IOdUI0D ap o1U3 99 oTJonbr] 
SULP SINSOUI eB] Op SNOssap-1d UONsonb vias |] “o1U93 
99 OP INOMPIAIOIUI.P SJUSUISIOdUIOD Op Wlp & WLE 
op jURAIOsgO ‘AdAING oINyIpusdxg [eoIpey, [euONeN 
R] B JUWUSLIOdWIOD np esepod un gnbrydde yuo ‘ajdurexa 
Jed ‘(9661) IX1q 10 [JouUeD ‘SIoquesyO ‘se9AzT9 
SUIOW yUoWaTQeIOgIdde jUOS sINa]eA So] ‘SINOMOIA 
-IQUI sop jUsWIsyOdWIOD eg] INS sapmye soNe.p ke 
JUSIUDATVIeAUIOD ‘sIojayno L janbidoosojgusew spueq 
INS SI9ISISOIUS JUSIEI9 SMOIAIOJUI sot onb yuoreAes 
SINIMAIAIOJUL SOT sno} onbsind sgasye JUoWWIeUUOIS 
IngeA ouN ajuasgidar Inb 9d — piepueIS MOIAIOUI Pp 
gouonbgs oun e sed puodsa0d ou Inb INaMaIAIOyUI Pp 
juouoyOdWIOS UN suroU ne Jed sayono} ywUOS s}USWIT9 
SOP UOIAUS %7ZZ ‘[eI0} NY “WWOUII}JDOLIOOUI SoyLIOSUI 
sasuodg1 sap 30 sajgjduioour suonsenb sanbjenb 
juoWaTes9 SAIOSqO UGC (‘«jsue ZE B VsNodg aNOA 
yo “*») asuodal e] 1OyNTd sIJLI9A seu ‘UOoTsoNnb eT sed 
II] OU INIMAITAIDIUL | ‘SVD Sap %¢‘¢ sueq (‘suoNsanb sap 
AIPIO | Ins s2oudLIgdxo sind] sULP SI9SITNN UO Q866I 
‘8866 JOAOW 19 BIOO| onb ‘jusWI9URIUOdS ‘saTeqIoA 
SUOISSOIAXO SOUIQUI SI] JUISSISTOYD SINIMOIAIOIUL SO] 
gnb 19}0U ap JURSSdIQIUT ISO []) “(«Lasnods 9}0A 39 “> 
a[durexe sed) juepuodai 9] 19 o]UeAINS OUUOSIOd B[ OUD 
Udl] a] JUeWOdWOD sn{NUINs anne uN yO NId WUINOJ stew 
‘uonsonb ey sed iI] OU INOMOIAIOJUL J ‘SBD SAP ABC suep 
‘snd oq “INoMaIAIJOUI.| Jed yuepuodel ne sgjuesoid 
sed JUOS OU SJUSUIZTS SOP %¢‘G 19 ‘sTanb spo} sn yUOsS 
sJUDWIa{a Sep %8/ onbjenb :prepueys somnpgooid xne 
sed yuepuodsai0d 3U INOMIIAIOJUL Pp syuoWeyOduI0S 
Sop JOAIOSGO Pp yoULIOd OgpIA Sepueq sap asfjeue,T “| 


:sduia) np Jouse3 op yuayjourod tnb (7) syuepuode 
Sap sJUaUIoOdUIOD ep 39 ([) SINOMOIAIJOJUI Sop sjUSU 
-9J0duI09 ep UOISIdA oUN eB sIdoId soUsLINIDO | op UOTE} 
-gidiajur anou aindde ogpta asepoo np ascyeue TJ ‘sduiay 
QUIQUI Us VSLUDUL NP SoIQUIOUI So] Sno} Inod oTuINOJ UOTeUI 
-IOJUI | DOVOLJJo UOSRJ OP JOSI[NN,P INOMOIAIOIU | B youLIed 
UOISIOA 91329 ‘agINP e[ op 3S9 IND 99 INO “SUOISIBA saIjNe 
soy onb sgnbiew snjd uodsey op onagid MoIAJOIUL | Op JOIASp 


OVI einpow un suep suonsenb sep aipso,| }@ Sues9e Sep UOIWdeduUOD e7 :syony O0€Z 


@ sjuRepuodal Sa] 39 SINOMOTAIOIUI Soy ssnod sweyy/eT[3 
UOISIOA eT ONb siNjOUOD ap jUoWoULIEd snoU sjeI[NSOI SOT 
"SQIZIP SJUDIOJJIP B JUOWIdLOdUIOD op dIUDB 9d JUISLIOARJ 
QOUSLI9AX9 93199 SULP IVSS9,| B SOSTU SUOISIOA SOSIOAIP SOT 
"«QQUUR SUIQUI B] JUBPUad SNUDA SNO} SALUOS sNOU» :sdurd} 
OUIQUI Ud oseUSU Np souUOsSJod so] sojno} op jafns ne uoN 
-PULIOJUL,] ITUINOJ B JUePUOdaI 9] JoUSW ynad vI90 ‘aUIgINXO 
seo un sueq ‘aliejuowgiddns uonjejnuins sues ourgu 
agudoidde asuodgi ve] yuRsstuINo} Jo DJURAINS UOTsoNb Pe] 
JULUIASP 19-IN[99 ‘“JUuBpUOodal 9] Jo][INSIe e UOSeJ op ayLOSUI 
919 & ajUapgogid uoNsenb ev] e asuodgi ey onb [eusis op 
stojied yas INdMIIAIOUI | Jed sonbsrewial op UOISNIOUI J 
(L661) PeIuoD 32 Jaqoyos suep uoNsenb js¢ [I JUOp 
IN[99 & B[Qu[quIVs JUBPUOdaI 9] SIOA 9IUDIIO JUSWIOVIOdWIOD 
un Jaj}dope jnod InaMmatAlowuI| 3a ‘nod snid juow 
-a[qeiogidde jso yuowlgta red giovsuod sduia) a] “uenbgsuoo 
Jeg ‘(euuosiadjusulg{9 UOISISA Be] Inod onb pues snjd juow 
-g[qeiogidde jsa ajonog op 394J2,[ ‘a[[1I3 e] op uoNdaou09 
RI Op osned & ‘slojajno], “oWgY) I] INS sgpuoj aIa}eIIS 
R] Op sosejurae sop oinsoul ougUI eV] Suep sed a1dIJQUOq 
ou suUOsIod/a][U3 UOISIOA eB] onb sIN[DUOD op jUsWeULIEd 
p UNS el op oInaLgjul sted vy] suep juLINSIJ syeqnsol 
SY]) ‘uonsenb ouigur ve] e BdeUSUI NP SoIqUIOW soINe .p 
Inod soruinoj sasuodgi soy ‘ajduroxe sed ‘sayjanyxoju0o 
SUOTULIOJUL soNNe.p WUINOJ 39 BUS oun syOdWO0S 
OVI 2npow 2] anbsioy resA yUSWIAIgITNOIWWed 4s9 BJaD “USUI 
-g]loej snjd s[nolep as MOIAJOIUI | 39 ‘UONsonb oureysolid 
PR] Jodionue yuaaned jyuepuodas 9] onb uarq Issne INIMIA 
-I9\UI J “jUepuodsl np 299 op SWUWIOD INIMIIAIOUL,| 
op ued be] op sayeqidA SUOTJUDAIOJUI.p suIOW oWOdWIOD 
esuodgi-uonsenb snssaod0id a] enb aylos ap ‘suonsanb 
xne juoniIqey .s juepuodgi a] aonb ustq Issne INOMIIAIOIUL J 
‘uonsonb e, Jojedo1 sues o8eugui np souuosied soy 
$9}NO} INod apod sWIQUI 9] JANUS .P SIOTe WJJNs [J “UOTeNIS 
91100 B Jaydepe s JUSWIaTIORJ Jnod INSMAITAIOIUI | ‘aNUOD Jed 
‘QUIQU} 9] INS 9puoJ BJQPOU! UN suRG “aseUSU NP soeIquISUI 
seine so] Inod uonsenb sugui ve] e sosuodgl sop uoyeoIpul 
QUNONL JURSSIUINOJ OU 39 JUSUIII,] INS sexe UeIDS.p UOT} 
-BINSIJUOS 9uUN INOd reJA JUSWIIEI[NOIWed Jsd v[aD “NesAnoU 
op uonsenb ey] asod jt sdwia) np yednyd ev] stew ‘uarAnos 
Ud,S INIMIIAIOUL | Slope *«jasuodg1 ouuog ev] tUINOJ efop 
B Juepuodg 9] Ied ‘uoNsenb 9320 sed ajedai ou» :sojodde1 
QS JOP [I ‘ayUeAINs oUUOSIod eB] JopIOge p JUCWIOWI Ne 9 
“UOTJBULIOJUI 9)}99 JOSUOUIDUI JOP INOMIIAIOIUL | ‘QUUOSIOd 
RI Ins sgpuoj uoNdsou0d eI Ns steUUONSeNb 2] IS 
*(cuonsanb 33399 Jajed91 
OUI OP ZISSIO» :alIp 3ndA Inb 99) «aguUe oWQUI eB] JUBpUad 
SPALL SNO} SOUTWIOS SNOU» :adA} Np So UOTORAI BT ‘(UOITAU 
%OE) seo sop uonsodoid a0} oun sueq ‘snssao0id 
np onbisoy] a19}0eIed 9] JUoWIOpIdel seq yWeuUOdsel juepuodel 
9] ‘osvugu Np souUOSiad sa] $ajn0} Inod suonsenb sa] asod 
uo | onbsioT] ‘auuosiod ek] INS S99PUOJ SUOISIOA XN JUSUIDAN 
-eIedwl0d MIIAIOIUL | B JUSUIUUAIOJIp yUa}depe s syuepuodg1 
S9] 19 SINOMIIAIOIUL Soy ‘OUIQY} ST Ins sdaxe sUOTIPUOD 
SOP SULP :s9dUdIJJIP Sad Inod sutour ne sayjated suostel 
Sap ITUINOJ ap stuLied B sNOU OEPIA SepuRG Sap asATeuR "J 


6cc0 


100'0 >see *10°O > dx 


eV ETE” kT kK B'T™ uahoul [ono ep 193g 
v'0- Gls €'0- £0 suuosied + jusui9|g 
iG O'€- Ge S‘0- suuosied + af[UD 
0°9- i © Soe mncrS sUIQU} + JUSWS| 
6'S- 9'r- y°9- 8°9- SURI} + STUD 
UOISIOA LT UOTIS 9JONOG ep 19;Jq 
y'€- Cts ave SZ aseusul np souuosiod 
SaTNe Sa] $9}N0} 39 
aigiuaid vy] aus souasgzIG 
9]9N0g ap IIT 
eee Ll xexl 9 xaxS'L «408 souuosiad sa] saynoy, 
‘ ‘ ; z aseugu np 
a av, Ls 99 souuosied sanne sa] SIN, 
‘ ¢ aseusul 
ve be 66 v6 np suuossod oJgiwissg 
juoulgy9 ed aging 
syUoUIg|9 
sq] snoy,  UORPMIS SPAT a3y juowl71g 


(sepuodas ua) afoNOg ap JajJa 19 daINq 
[ neaqu 


(‘sayono} arg sed JUSIvIASP OU S}eI[NSEI Say anb 
OOS Op ‘SUOISIOA $9] $9INO} INOd sWIQUI 9] aINQ WeIASP Jaljo 
309 IQ “JUSWIYTE JoTUIOId yn0o} 9] Inod jUsWIEpUd INI] JossTeq 
oirey nd & Inb 99 ‘Juaresi[yN s[t UOIsI9A oT[anb sed yuareaes 
OU SINIMIIAIOJUI Sop aonb JajjJo SUID Nod ve doUSLIEdxe aTOU 
ap uondaouod ev] aonb gIqIssod js []) ‘puess snjd ajonoq 
op JajJo UN dUOp JUdLOdUIOD 39 ‘aseUSUI UN ,p sajUanbesqns 
souuosied saz 39 ouUOsIod owaIxnep ev] Inod snssad01d 
9] JoIg[9008 Pp JUSWOULIOd SUIQY) 9] INS SdgpUOJ SUOISIOA 
SoJ ‘IsuTy “(| nesfqe}) suUOsJed e] Ins 9xe suOsonb sop 
dIpJO UN JUDAINS INb SUOISIOA Sa] INOd }sd,] OU [I,Nb surey} 
9] INS S99Xe SUOISIIA SO] Inod puvss snd yuowsqeidoidde 
4S9 JJONO ap JOJJ9,] aonb 19}0U op JURSSdIQIUT 3S9 [I ‘SIOJaINO J, 
‘(L661 ‘[]09 18 Jodnod) STHN soeseusul sap soiquicul 
Sop ISI] BP] & SO9I] SoIMag}Ue sooUslIgdxe sou suep 
g[NO| aP S}aJJo SOP 9AIASQO SUOAR SNON “soUaLIgdxa 3]3990 
e aidoid juowolginored sed 3so,u afonog ap yoyjo.'] 
‘({ negjqe}) suuosioad aigtwiaid e] & JUOWIOATILIOI 
UOIIAUS Ye op uotonpgs oun aJuesgida1 inb 90 “uaWIgTS 
Jed sapuosas p‘¢ [e}0) ne sUUOp UdAOU Z[DNO0| ap 39JJ2,T 
‘sajuonbgsqns souuosied xne jJuowdAneseduios oseusul 
np suuosiad ssgrwiaid ey Inod syuswouslosual Soy II[[IondeI 
e sduia} op snd dnooneaq a1desUuod INOMITAIOIUL,| :JUBAINS 
suguiougyd 9] WWDep UuOIssaidxa aneD “(TOOT SYony) efonog 
op 12JJo,] aJedde uo] onb aqeiogidde jayjo — o8eugwi 
np sajuenbesqns souuosiod say 3a ouUOsiod suIgrIxnap 
RI INS SJUSUIOUSIOSUdI SOP 9}9a][09 ve] Inod opides yUSWIAIIT] 
-nored jsa UOISI9A 93309 anb onbiput agi[Ieyep asATeue our) 
(epider snjd aj[o-jso oueyi/a[U3 UOISIOA ve] lonbinog 
(‘(a[qetogidde uonoeiajut sunone 
‘[210} 19JJ2,| Op Sion) xnep ‘[Q0‘O > d :suonsanb sap oipio0 
‘[210} 19JJ2,| Op S19) uN ‘[0Q‘Q > d :suvio9 sap uoNdaou09) 
sduia} np uonnumuip ey juoWUWepusdepul yUosLIOARy — 
suoljsanb sap dIpso ,| 9 SURI Sap UOT}da0U09 BI SIIP-k-1S9 9 
— sinojoerj xnop say aonb onbipur soured ev] op asdyeue 
oup) ‘juswIg[9 Jed sapuoces ¢‘¢ op INna[eA oUuN oUUOp 


000g aiqwessep ‘a}enbua,p sanbiuyoe | 


32 ‘OWIQYI/9[Ud uOIsIOA eT B onbi[dde.s guiquios jaqJ9,7 
‘quuosiod ke] INS Ssdgxe SUOISIOA xnap se] onb sspurow 
yuowle[qeiogidde sduzo} un jussoddns o][113 ve] Ins ssapuoy 
SUOISISA XNOp So] 39 ‘oUUOSIOd ke] INS SagpuoJ SUOISIOA 
xnep seq onb sazinos snd juowsaqeiogidde yuos sway) 9] 
INS S99X¥ SUOISIOA XNOP Sd] :SJULAINS S}e][NSEI Sa_ SuOUD}GO 
snou ‘sinojoRJ XNop so] sUONSUTISIP snou Ig ‘aY4oR) eI 
Jjdurosoe & stu sduia) np UOTNUIWIp Pe] sup sTOI UN Jonof 
UZ[qUIEs sINd}dej XNap sot onb JoUUOTUSW op oWOduN TT 


UOISIOA P| UOTAS JUSTE Jed soinq *¢ saNSIy 


loo>d UOISIOA 


ewey) + e6 ewey) + jUeWE}9 eULOSIEd + oJBUUOSIEd + juOWO|9 


o 


+ 


sepucoes 


w 


"solgtuoid xnop say aue 
JUDIENIS IS SUOISIOA SOIINE XNIP SIT “dUIVYI/S| [113 UOISIOA 
RI Inod uoIMAUS %/]{ Bp UONONpss oun xT 3S9,D “OWI 
/e|[B UOISIOA eB] SUeP SepuOodes ¢‘¢ gsoddns e jUOWIZI9 
onbeyod ‘anuoo Jeg “(pIepUL}S UOISIOA P[ SUTUIOD SgI9PISUOD) 


QUUOSIad/JUSUIZ]9 UOISIOA P| SURP JUdUUgTE Jed sapuodes 9‘9 
QIOVSUOD JUO SINIMOIIAIOJUI Sd] :SUOISIOA oenb sap 
sunoeyo nod sjuSJEJJIp JUoUTaTQeIogidde 919 v sgINp eT 


SLVLTINSIA ‘bp 


“sduiay 
Np SoINsowl xne sogUIqUIOD 919 UO sajyURI[NS9I saguUOp 
S93] 19 ‘“JUepUOdsI Np Jd INIMOIIAIOJUL,| Op JUoUIoOdUIOD 
NP UONIUOJ Ud S9POd 919 UO OEPIA sJUBUIZES Sa] ‘“aseUsUI 
NP SOIQUIOW Sop d)SI] kT Op UONIS k] IINOoJed op ulen 
Ud INIMITAIOUL.| onbidoosojguseul opueg Ins asiZolue 
SUOAR SNOU jd ‘pseseY Ne SdISIOYD 939 JUO SMOIAIOIU! EZ 
‘snd oq] “SMOIAIOUL [QS Sep UOTIDasS aed ap jUOUIgT 
onbeyd & gidesuod sduia} 9] onb sure ‘aseugui np serquicut 
Sop 91ST] ve] Inod stnbal [e10} sdura) a] ginsou sUOAR SNON 
‘sjuepuodal xne 39 SINOMOIAIOJUI XN OUOP 39 — MOTAIOIUL 
onbeypo & aloyegTe USE] Op saNqLe 919 B SUOISIOA aeNb 
Sap UM, ‘OVI Salgnbus sap sonbrydeizourgp-o190s s}aTOA 
xne JuoWsTfaniiqey senbrydde semeuuonsanb sap uondeou0s 
RI aUasgidal a][o — PILPUL)S UOISIOA LT SUIWIOD agJEpIsUOD 
1So QUUOSIOdNUSUIEIE UOISIOA BT ‘(7 BINBIJ) snssap-19 a}LOep 
anbiu09gy} a13ajeNs ve] op UOT}OUOJ Ud suOTJsoNb sap aIpJO | 10 
SUBION SOP UOIIdI0U0D PY SIJIPOU SUOAR SNOU SIU “INAMOIA 
-JO\UI,| Inod suononsuI sougUI xne Ja sUOTsoNb sap sITOqT] 
ouigu ne jodde 31eJ yUO SUOISIOA SOT SOINOT, “OWIVUI/ST[I3 
UOISIOA OUN Jd SUIQY}/JUSUIST9 UOISIOA OUN ‘oUUOSIOd/aT[II3 
UOISIOA BUN ‘UUOSIOd JUSUIZ[P UOISIOA DUN :o[NPOU o] suep 
SaSNpOUI 939 UO ‘auUOSIEd Jed s]USWUETE SION DeAR ‘oseUSUI 
jjed uN,p saIquUIOW sop d)SI] OUN,P SUOISIOA send 


(uei99 un ajUasgidar saseo anbeyo) souatI9dxa | SUBP SODIJUIDA SUOISIOA O1eND *7 ANSI 


omegy/emAD 


92 GI/349 M9 


suuosi9d/e15) 


OV] ainpow un suep suonsanb sep eupio,| 39 SuBID9 Sep UONde9dU0d e7 :syony 


euuosisdjuem91y 


Bec 


“SOINUIW €7 9INP JUO SMOTAIOIUI Soy ‘QUUSAOW UY “SOWIE 
SIOAIP Ins sagsod 939 UO suoTsonb G6 onbjend “¢ID OVLI 
SUWIeIZOId Np sIpPed ds] SUBP SOQUDU 919 JUO SMOIAIOJUI 
SY] ‘a[qQIeJ Zasse 939 & UOTsanb ouN ze dsuOodgI-UOU BI 19 
‘Op op 91a & osuodal op xne} J] ‘sIoj ¢] e,nbsnf suoydgyay 
Jed xno dane gnbrunuww0d & UO Jo ‘2dURAR | B UONONPOHUI .p 
ae] oun sjuepuodg: xne IudAIed ye} & UDG “SMOIA 
-IQUL [QG=U QUO JUO SINIMOITAIOIUI SI] ‘QGGH[ SICUL QZ 
ne nbsnf 39 ‘g66] JOLIAST BZ np Jaiduioo y ‘anbnertaos 
uOIUA) suUTOUR | ap 19 StURINOY ej ap ‘eUZOIOg BI op JUeU 
-dAOId SpULUAT]e OUIZLIO .p SJULISIUIWUI Sop 9SOLI9}UI & UCD 
"866 SIC US OUSeUIO]| YY Ud dg[NOJEp 3S9s BDUDLIOdxo TJ 


AGOHLAW ‘¢ 


‘yuepuodal 9] 
10 INIMITAIOUL | JUD BdVdIJJO Syd ey UOT}OVIOIUI | aIIOSNsS 
dWI9U} UN INs axe suOT}sanb sep s1pJO UN .Pp Ja a][Us oun Ins 
d9Xe SULIDE SAP UOT}dadUOD OUN Pp UOSIeUIQUIOD FR] :oJURAINS 
asqujodAy | & Nal] QUUOP JUSTIeAR dINILIQ\Ue BYdIOYIEI PI 
OP S}eI[NSPI SI] “MOTAIOIUI SUN Pp BYdIeU QUUO P| SIINEp 
no JasuoAeJ yusAned ‘suonsonb sap sIpJO., 39 sueIOg 
sop uoNdaou0d PY SIIP-k-1S9 9 ‘INdJeUIpIO Jed g}sIsse s[NPOW 
uN .p sjoodse xnap SA] ‘MITAIOIUI | OP [RIOOS 9}X9JUOD Ne 39 
areuuonseanb np uondasuos P| & dat] 3S9 S][o STeU ‘UOTFRW 
-uviZ01d e] ap jUaUIeTNas sed dAgIaI OU OY] F[Npour un .p 
UuOHRST[NN .p ayploey ey syns Inb asgujodAy | gidope suoae 
Snou ‘altej 9d INOg “UIe1I9} J] INs sdUaLIgdxo ouUN suep sajuR} 
-[NSQI SUOISIOA aVeNb So] JOIJUA & USE} Op (SWAY) 9] INS 
9Xxe BIPJO Jo QuUOSJad I INs axe aIpsO) suoNsanb sap saipio0 
XNOp Soy 39 (A][1I3 e] Ins sapuoj uoTdaou0d 39 JUDWIZII | 
Ins sapuo0j UONde0uU09) ULIDE Pp SoTepoOU XNap Se] UIqUIOD 
SUOAR SNOU :d]][1I3 DUN INS sgpUuoj suvIDg sap UOT}daDU09 
RP] op Jo onbyewsy) a1s9ye1)s e] op sasejueae xnev jadde yey 
SUOAR SNOU ‘IdI suOJUSSeId snou aonb adUaLI9dxa | sued 
“Q[QPOUL 9d JUSWA}j9U JUDIOJOId 
SINSMIIAIOJUL say ‘sdwis} ep suloul jusWslqeiogidde 
puoid [I 30 ‘snjol op yo suondnuajul.p sutoul ‘agaoj9 
sulow oasuodgi-uou oun syoduios onbyewsy) e[epow 


3] :onbneurgy} uoNdaouod oun jusWaYOJ JuaIndde IoAOPy 
}9 QIOOJ OP siei[Nsei sa] “ojIns op Isure yo ‘souuosied so] 
$ojno} nod uonsanb surarxnap vy asod uo sind ‘sayqissrupe 
souuosiod sq sayno} Inod agsod 3sa uonsenb sigmueid 
RT ‘onbyeusy) aisayens ey gjodde ‘ajepow sulerxnop 
a] UOTaS ‘ouUOSIed RI INS daxe aIdQ}eNs ev] a[odde uo] 
anb 99 jsa suorjsanb sap o1pso Ja} UA) “aJURAINS oUUOSIOd PI 
e assed uo sind ‘a8eusui np ajqisstupe ouuosioed orgtuia1d 
eR] Inod suonsenb sey sajno} asod uo ‘ajgpour sJonueid 9] 
UOTIS “(Q866I ‘C8661 JOAOJP] 19 DIOOJ]) OSVUSW UN ,p SoTqIS 
supe souuosiad say $ojnO} INS SUOTVULIOJUI Sap II][IaNdOI 
® JUvAJos suoNsonb sap saipio xnep ins sgueW 90UdTI 
-9dxa 9UN,P syey[Nsa1 say gJUeseid UO IBAOP, 19 DIOOJ 
LOBeUDUI NP SdIQUIDUI Sa] Sno} Inod sjUsWIsUSIOsUaI 
SOP IN][TaNdaJ INVA UO,] Is sUONsoNb sap aIpJO INdT[IOW 9] SO 
Jenb ‘inajeurpio Jed 9}sisse JUOWIOUUOIIAUS UN,p assadnos 
op onbueul 3] NA :a}UvAINS UOT}SaNb eI Josod as sioye nod 
soleuuonsanb sap uondaouod x] e& assaigyut s nb suuossed 
auy) ‘sdura} suIgUI Ud BBeUDW Np SoIquISU sa] sno} Inod 
sogtidoidde saseo say JOY9O9 JUSUETIORJ Inod INSMIIAIOJUL | 
‘g]duioxe red ‘«stou sno} sauTWOS snou» adj np asuoda 
oun Inog ‘juepuodel 2] Jed atuinoj oireyuowgzddns uon 
-PULIOJUL | JUOUIOJLIPSUIUT JasI[NN Pp INoMIIAJOWUI UN Inod 
g[lovyj yso [I ‘surtidut oreuuonsenb un,p souasgid uq 
(Z661 [[ouURD 39 IXI[g ‘oqoog “S1aquesyO) 
aoel op uonsenb jsa [I,nbsio, «siiou sno} sowUos 
SNOU» :JoIe[oep Issne elinod juepuoda 9] {«z7E B ua asnodg 
UOUI 39 sue p¢ Ie ,f» :axouUOD UONsenb oun eR yUOUTATe9 seul 
‘(«sue pe Ie ,{ ajdwioxe sed) aasod uonsonb e] e& asuodal ke] 
JUSWIZTNIs UOU assTUINOJ JUepUOodaI a] anb (snossap-I9 BLIOA 
J] UO DUS ‘JUBANOS Zasse SUIQUI 39) SALE [I ‘aseUgUI 
NP SoIQUIsW op ays] oun Inod oireuuoNsenb 9g] yWYduIeI 
uo nbsloT (‘sdos9}eo-snos sap sUIUIOD sadgzIel] 1UeI9 
s9iUS sane saq No sauuosiad se] ‘satoseye9 soyedioutid 
So] JUoUesgIdo1 saseUusUI So] :sagUUOP ap saseq sap 39 
sonbnewwojur soummueisoid sap yuaWaUUOT}DUO}J 9] Jed otyred 
ua onbijdxa,s ejad) ‘asnods uos ap 338] @ Jassed op jueae 
suonsonb sane .p 32 gilIe[OOs ap nesalu uOos “UepUodsI 
np 238,] apuewep uo] anb sa8Ixa yned OY] 2FNpour 9] 
‘ISULY *(966[ JOAOJ] JIOA ‘«Sagdno13 suonsonb» sap jafns ne 
‘LOOT SYON- 19 L661 “[]09 30 Jodno|d MOA ‘«auUOsiad ey Ins 


I [UB e] Ins s9puoj UONd|duU09 Ja USUIII,| INS s9puoJ UONd0UOD *] eaNsIYy 


giueuig[g = ¢ UAT p 1USUg|9 


€ 1UsUUg]9 


Z jUSWIgTP | JUsWI9|9 
[ uelo9 


oy[us 


Lee 


9 1UOWI9|9 


¢ 1UaUIg|D y JUOUs|9 


9 ueI09 ¢ uelD9 p uelog 


€ 1UdUIg|9 Z UOWIg|9 


| 1UOUIgTP 


€ ueIO9 Z uvlo9 


| uelo9 


qUsUIg|P 


0002 aiquiecap ‘ajenbusa,p sanbiuyoa | 


g9xe UOd99U0d» eT op jofns ne) aURAINS oUUOSIOd FI B 
suonsanb souigui se] esod ou InaMotAroju | aonb jueae 
sogsod juvj9 suuossod oun & sagssoipe suonsonb soy 
$91NO} ‘“BSVUSUI NP SOIQUIOUI Sop 9)SI] BP] B SUONseNnb sap 
sipso un yuonbrjdde Oy] se[npoul sap yednyd ey :19]J0 
309 UsIq oNSNII[I Inb ojdurexe un 1910 A. ‘suolsenb sap 
oIpJO | B JUeNb ‘InoMoIAIOWUI,] Inod ‘assatdnos ap nod 
$91] 39 SUOTJSONb Sop spISII SQN) SIpIO UN o}[NS9I UA II 
‘QIJU0D Jeg ‘SJUSUII[P SOp UOTNIEXA | B IU) Ud,s nod 
y0 JOIONOS Ud,S B Sed BU INOMOIAIOIUL (J ‘27a ‘suonsonb 
Sop JUSWIOUTeYoUs | ‘SON[IJ Soy ‘sJotpNoed syuepuodsl 
Sop B SegUISOp suOsonb so] ‘JUSUIS[NOIJEp 9] SIAINS 
OP INSMIIAIOJUI | Op wed YI op sI[NOIJIp 33n0} aI1Ag 
Q][9 Jed osnosejuRAe suosonb sap sIpPIO.| op SNpIsu 
91199 9ANOI] UC ‘jURAINS UeIDE | B ossed ou oUaISAS 
9] onb jueae uonsenb onbeys snod [19}u9] ayono} ev] 
Ins soAndde ynej [I ‘UOWaTeULION “2dURAR pp QUTLA}Ep 
so suonsonb sop sIpsO.] ‘sanflJ op 32 suoNsonb sap 
JUSWISUTeYoU | op osesn JUsWo|dwe aIIej yUoANed OVI 
SO[NPOUI Soy Is BWQ|y ‘suONsonb op Jaynes asgNns yned 
OU INSMIIAIOUL | anb sa OV] S[Npouw np sesejueae sop 
UQ ‘essadnos op onbuewl 9] 1sa INdyeurpio Jed saysisse 
MOIAIOUL GUN sUep soWg{qoid sap Josod ap onbsu 
Inb onbnsigjoeseo ourgixnap vy] :assajdnos op onburyy °Z 


{29Inp eb] ap 3so Inb 90 inod aseusut 
OP SOIQUISUI 9p 9)ST] SUN .P DOKdIJJO | OSeIULALP JoIOIOUIe 
9 UOTEJUOWIZes ev] oINpgI Inod oIeJ ond :uonsenb ey] 
Josod ouop jned uKO (266 ‘1109 30 Jodno_ suep sjieiap sop 
BIOANOI} UO) SY") PI JOINIOX & stu sdus9) nv JUSWISANLIAI 
Q [U3 oun Ins sgpuoj uoNdsou0d ev] Inod tndde sfqrey un nb 
91¥JSUOD SUOAR,U SnoU ‘aUOD Jeg “o[[U3 oUN Ins s9puoJ 
uondsouos ev] suep a[loey snd s[quias age ud [NdeI 
UN sUIgJI “oJUEpgdeid UoNsenb ve] ap ssguuOp so] JURALIOSUT 
ud JNO} aJURAINS UONsoNb eT oI] e JadUsUTUIOD yUaANed 
SINIMOIAIONUL SIT “‘UONLIUSUISAS B] SIINpal op JouLed oT[U3 
RI Ins sgpuoj uodaou0d eI anb gAJasqo SUOAR SNON 
‘OVI S91 
Inod d][Us v] Ins s9puo] d][99 9 JUSWNII | INS s9puoj sULIOD 
sap uoNdaou0d e] aNsNIII snossap-19 onbrydeis sq *(set[on) 
-X9]U0D SUONPULIOJUI JO UOTINIPXS,P ISSO}IA :soseJULAL) 
uevlo9 suIguI UN eB UOseNnb suN,p snid MOAR Pp INAMIIA 
-I9]UI | & youlJod ayja ‘snid aq “(L661 [109 10 Jodno_ suep 
S[le1ap Sop PIDANOI UO) saplovj snid jos sIQLUe Ud S[NdII 
soy onb tsure inof e@ sastw sa] 39 ‘o[NpOwW np s[quIesUS ,p 
ONA JINI|[IOUI 9UN INIMIIAIOIUL | B QUUOP aIJe ‘juUepuodsl 9] 
Jed Istoyo d1ps0 | SUBP UONPULIOJUL | JONSISOIUS ,.P SINIMIA 
-IO\UI xne JouLIad aI[q “OULIOJ BP] INS 9x o[NPOU UN gITTe9I 
Ud Sd dI[US 9UN Ins sgpuoy UOTda0uU0D eI ‘ued es INog 
‘URINY,| INS sgpuo] aId9}eNs ouN INOd (Qg6I) S[[OYSIN 12 
asnoyx Jed saytioap sonbysugjoeseo xne puodsaL0d soueld 
-9dX9 a1]0U SUP SdIJLIDA JUOUIIIO | INS BOX UOISIOA BT 
‘gipso jenb oyioduit ,.u suep sJt[dures so] 39 
anne | e dureys un_,p sessed nod Inosind ap sayono} sap Ins 
JoAndde yned inamatAsoju J 39 ‘a][U3 uN suBp NO Neeqe} 
un suep sduia} swigUI US SpjUasgid yUOS S]UAWUgTS SInoIsny{d 
‘QULIOJ B] INS sxe uvIOg UN,p Iso IND 99 INog “jUSWIgI9 


OVI eInpow un suep suolsenb sep apso,| ja Sues9g Sep UOI}deouOd e7 :syony 


anbeyo sgide sagyndgxe Juos sanbi3oy suoneiado sa] soyno J, 
‘saffenuanbgs sosuodal sap juesoddns sjusurstg sinorsn{d 
adnoisel ueid9,] Ins 9xe s[Npow up ‘sWeWI UONseNb 
PR] INS INOMIIAIOIUI | Op UOUNHNe | os8UIp UoNdaouUOd aIIA1 
oun ‘JourUeIsold eB a[19e,J “one | B ULIO9 UN,p UOTIsUeY 
R] JUeINP Nat] UO sonbiso] suoyeIgdo soy Jo ‘stOJ e] B SISTeS 
ap dureyd un 39 uonsenb oun Jayorye Wey yUSUIZTS,| INS axe 
Q[NPOUW UL) *SULIOJ P| INS d9xe JO UBIO | INS 9axe “USUIZTS,| 
Ins dgxe :INJeuIpJO Jed sgjsIsse sojnpoul sop nod sueso9g 
Sop UOIdsdU09 op saIsgyeI)s SION ILWOEp 1UO (QR6]) ST[OUSIN 
9 9snoH{ ‘s9][Upenb oy1p uoydeouos oun 39 onbyeuwsy) 
op uondssuos sun greduiod suoae snonN ‘uoNesTTNN,p 
S[lovJ k] Op 39 oMTeUUONsoNnb a] Wjdurer e stu sdwia) np 
UOOUOJ Ud BITOJeIOQR] Ud essa] & SOSTU 939 JUO SUOTSaNb 
OP SLIQS OUN,P SUOISIOA xnoq ‘SogUUOP so] JPALIOSUI 
Ud JNO} 39 JURPUOdEI 9] DdAR UOT}ORIOJUI | JUBAIJOSUOD UA 
nO} ‘elreuUONsenb 9] sIpusidw0d xno ap INAMIIAIOUL | 
@ JouLed inb oset[nored ueis9 .p UONeINSIyUOS ouN INaTeA 
Ud STU JUO ‘(966 SYON pue Jodno_d ‘ussueH 6/661 “110912 
Jodno_d) s3eUsUI Op SOIQUIOUI op dISI] DUN Pp SULIOS Sap UOT} 
-BINBIJUOD L] INS SILOWIOQE] US S9QUOU UONESTTNN.p syloey 
Op sIessd.p SdLigs sInaisnjd ap snuaygo ‘s}eyNseI SON 


(P86 ZIOMOTHIRI] 19 SOAOID CQ6T 
asno}) Saitepo sed ang-jnad jos au sa][a aUS suONRIaI 


S29] 19 suoNsanb sop sousuted eT anb 49 «a;quiasud .p 
ona» op sed &.U INIMATAIOJUI.] onb Janbsreusal ye} uo 
*‘91U09 Jeg ‘MOIAIOUL,] Op JUDWIITNOIJEp ne no saguuOp 
xne JNU UOTEJUSUIZES Op o1Ue3 90 onb aaNeyUeNb uol} 
-BOIpUL SUNONR dJONUd 9}STX9,U [J “[19}Ua] soyono} sop 
Jed o9}UsWIBas so JURPUOdgI 9] 19 INOMOIAIOVUI | WU 
UOTOeIOJUL | ‘aInpgoo1d ap o1Ua8 99 sue “sdura} Suu 
ua sayoe} xnep sop Jadnoo0,s sed juaaned ou sjI — 
[10)U9] Ins JoAndde ,p juvae ajuRAins UONsonb eI sll] & 
JgouswUI0D sed yuaAnad au sj 39 ‘ajUapgogid UoTsonb 
RB] & Sosuodal Sa] JURALIOSUI Ud 3NO} aJURAINS UONSoNb 
RI Jadionue ja jueAe ud Jopiesol sed jusaned ou 
SINIMIIAIOJUL SOT ‘Od],] & JUOWOATFBIedUIOD ‘o]UeAINS 
uonsonb ey 30 ueld9 uleysoid of anperedde ojnsus 
0A 39 [19}U9] Ins oIndde ‘soguuop se] enue ‘asuodg1 
oun y104Se1 ‘uoysenb sun i] INOMOTAIOJUI UN UO 
-O[CULION “MOIAIOVUL | JUIN Nok] v oIsTes op snssodo1d 
3] :alreyuourgddns nevopsej un JosOdUult JOA 9S INBMOIA 
-I9Ul | ‘OVd] JUSUIOUUOJIAUS UN SURG :UONRIUSWIZIg ‘| 


‘INIMAIAIOIUL | INod assajdnos op onbueur 9 (Z 
J9 MOIAIOIUL | OP JUOWIOUTUTOYD Np UOTEIUOUISes BY (] :uON 
-eSI[NN,p gy[IOeJ Vl op syoodse xnop e InowNs syono} onb 
-LIOQY} 2OUSSIBAUOD ap juIod 9] ‘asAyeuL aJUasgid BT SUR 


ANOIMOAHL ALXHLNOD “7% 
‘(QO0Z UasULH 10 Jodnog| ‘syon,J) «aIdo[OUYIa} B] BP S}aJJo» 
SOP UOISSNOSIp 9jU999I 9209 op atyed 3ej JUoUUNSOp juasgid 


IT ‘(2661 ‘109 32 JadnoD suep ssgyjuAs sun eIOANON 
UO) UONRSTNN,p s[lovJ eV] Op 39 SMOIAIO\UI Sop ogINp 


9c¢ 


‘KueuLlay ‘NeISYOIY [LOS8 ‘9Z sseNsuayseO ‘AZOfOII0¢ Jo JusUIpIEdag ‘NEISYoI_ Jo AJISIOATUL) DITOUID ‘syn yore] “IG 


RI op ‘sjuepuodal say 39 SINOMOTAIOVUI So] Jed uONRId|N9R || 
op 9iel yuo sajucog1 snd sopme soq (Z66I SXOOM 
SUP 19 686] AIeSOY 19 SAAOIDH ‘IodnoDd suep saseyjuAs 
SOP PIDANO] UO) [SIDIZO] ap jo [aLIgyeW op sowigqold 
sop ins ynoyns gyod ke anbisojopoyjsu syosayool 
Rl ‘INQSP NV :SMOIAIOJUL SOP JUSWII[NOJOp I] sUeP SdIIe} 
-usuig{ddns sauig{qoid sureyico yINpoUI JUO InyeurpsO Jed 
SO9}SISSE SMOIAIOJUT SO] ‘9UOD Jeg ‘UONe}da008 | op SUIOW 
np no a1yyeduiAs op dnooneaq juajsojiueWl (766 JOP) 
sjuepuoda: so] onb owgw ep (p66 UNg Je Jadno|) 
SINIMIIAIOIUI SIT 30 (QGH] MNITOp 39 S][OYSIN) sosejuRAe 
Sop JUIN US UIA} 9] INS SINaOeIIP sa] Jo sINaYyoIoyo 
SY] [la Ug UN,p aNA 1S9 OVI, | & eigues uoNIsuey ey] 


‘jeued red sapnyd say Inod ajuessargjul yUoWoIaTTNITWed 
UONIUOJ ‘MOTAIOIUL | & SOUIO}XO SOQUUOP Sop 19139}UT,P 
OVI.1 2p gwoedeo ev] ap Jaroljougq juoredsa sjt — 


‘saqiosqe [aId1s0] 
OP 19 [OLI9}CUI Op xNLI}IUT $]NOD Say SIOJ oUN aygnbua.p 
sjospnq So] dINpal 39 JUSUIIIIeN-}SOd Ne 19 MOIAIOJUL | 
B JUosIe pp 39 sda) ap suloul JoJORsUOD jUsIedsa sjI — 


SINOMOIA 
-I9}UI,| @ NPUI neapsej uN Jasodwi sues saxajdwios 
snjd soieuuonsenb sop MOAd0U0D ap 30 UO 
-onbrewiojne suonsenb sa] Joureyous .p jouwwied OVI. — 


sop uondsouod ‘ure1a) a] Ins souaLIgdxe ‘uONeSI{NN,p sy[loey op ress ‘inayeurpso Jed ag}sIsse MOIAIJOU] 


‘ap NoeIsyoto-ny © syony Yorewl :onbruoyoej9 JowIND 


I 


‘MOIAIOJUL | OP SINOD N SOT[VAIOJUI Sop 
32 VOUdIZYOO BI ap SogiodooUl SUONLOIJLIDA Sop B ddRIS 
ainatiedns guyenb ap seguuop sop sN[Iensel yuaredse sj — 


(OVI. op aiuendod ajduuts ey onb syueyodut surow 
JUOS S}UDWINSIe sapljos op aonb stojsed a[quuas [I ‘stoJo}n0}) 
suostel sinaisn|{d inod Inoyeurtpso Jed a9}sIsse MOTAIOIUL | & 
UST} 9s saygnbus sap sajqesuodsal sa] 39 sINDYdIOYO SI] 
‘quusadoine syoIJayoal P| ap [eUIO JuIOd 9] sed 
juajuasaidal ou suQUIOUgY 39 op sonbIsojopoyjeu s}oodse 
Sd] “JUSUIOW 9] INOd ‘Is sWIQUI — (8661 dHOS eddnisjyelo1g 
‘L661 UOOY 19 BLINK] °C66I “PO6T SYN ‘1661 Prouyss) 
Inayeulpio Jed s9\sIsse MOIAIOIUL,] SIOA UOTNTOAS oun 
dAJasqgo uO ‘adoIng Ud Wg “SOUTULIO] efop 3so UOTIISURT) 
909 NO ‘OVI,] & Jossed op uley us yUOS stuP-s}eIq xne 
saqueyoduir sajejuawiauraAno8 saygnbue sineisnig (OVI) 
Inajeulp1o Jed s9jsisse MOIAIOUI.p sonbruysa} ap spre ,] 
B SaQUOUI JUOS sapnig,p snjd ud sn{d op ‘sayjauUOsed SMOIA 
-J9]UI soq Inod anb uarq issne sonbruoydga) saygnbus soy 
Inod (8661 ATHOY.O 32 STIOYSIN “UURY] “YWeID “WeyeTpieg 
‘royeg ‘Iodnod) psepueys sjgnbue .p onbruyoe} oun JTusAep 
OP UII] UD JUOS INdJeUIPIO Jed Sag\SISSe SMOIAIOIUI SIT 


NOILONGOUINI 'T 


‘suonsonb sap aipso ‘surg 
‘SATO SLOW 


‘d9INP E] Op 3sa nb 90 nod sinaMdIAIOU! SOP JUDWOPUI INI][IOWI Ne Noi] ouUOp 39 JURBPUOd|I J] INS 9xe JUOWIOVOdWIOD UN 
INIMOIAIOJUL [| ZOYO OyT[LoeJ SoT[Upenb onbyeuigy) uolsisA ey onb Jusnbipul saguuop saq ‘sjuepuod9y sop Jo SINOMOIAIOJUI SOP 
siortnotyed syusuayOdul0d sap & SdTGeNQUIe JUOS SUOISIOA SO] 91]US S9DUDIOJJIP So] onb juonbipul ssepoo op saguuop sep 
S911] sjey[NSgI sa] ‘oIreEuuONsonb 9] 31[dual e stu sduia) np iso Inb 90 snod sapides snjd soy syeq[Nsg1 so] SuUOp onbyeW9Y) 39 
a9 {[Upenb uoIsJaA eR] ‘Jatpnosed u_ ‘sINOMIIAJOJUI Sap JUSWISLOdUIOD J] 39 MIIAIOIUL | OP VoINp k] Ins a[qelogidde souonyur 
aun uo suonsenb sap spso,| anb ouigui ap sueio9 sap uoNdaouod ey aonb juanbipur syeqjns9i sa] ‘OVI,] op uondeouos 
ap sonbysug}oevsed sayUdIIIJIP Sop UONKSI[NN,p 9I[IOv} e] QNTeAg SUOAE snoU ‘sagUUOP Sap opie], YW ‘osePpOd op euIgYoS 
UN, Pp UOTOUO] UD sagsATeuR sind ‘anbidoosojgUZeW spur INs sagISIZoIUD 919 1UO PIeSeY NE S9QUUON}DII9S SMIIAIOIUL PETZ 
19 ‘OV IL] ouIWeIZ0Id ne sagIZ9}UI 939 JUO sduId} Np soInsoul soq ‘sjUepUodgI sap & 39 SINOMIIAIOJUI SOP & oI1O}egTe 
UOSey OP S9gNQUIe 9}9 JUO SUOISIOA aTVeNb say] ‘7XZ I[[aUW0}9e} UOTda0U0d dUN sup sUONsaNb sap sdIpIO xnap jo sueIO? 
Sap suotdaqu09 xnap JOYLIZA op ue sagredu0d 939 JUO IBEUDUI NP SOIQUIOUI Sap 9}SI] JUN ,P SUOISIOA oeNb ‘uoHONpold ap 
apnig 99399 sueq (10S = 4 ‘%pg esuodal op xne}) suseUIOT[Y UD s]UeIZTUIUIT Pp seidne s9uaUI 919 & OVL] 2d} ep ajgnbus 
9UN ‘866 SIeU! U_ “Ule19} J] Ins soUdLIGdx9 sIQTUIOId OUN Pp S}e][NSOI SOP SUOAR SNOU STeUl ‘aJI0JeIOQE] OP SUOTIPUOD sap 
SUEP SOQUDUI 939 JUO SIDUDLI9X9 SID “INIMIIAIOJUL | OP [TART] 9] DII]IOBJ] xNeaTQe} Sap kB Jo Sa][UB sap ev sINOdaI 9] onb gnuoW 
UO So[eJUSUIDUIOANOS saygnbus sajyueWOdUII XNIpP op SoseUDUL SOP SOIQUIDU Sap d}ST] BP] INS SA9pUOj UOTEST[NN,p oytloeJ ap 
soousugdxd saq ‘OVI 2[Npoul 9] suep sagsodsoout juos sogudosdde sanbysug}oesed 39 SUONDUO}J Sap Is DDBIIJJO J9 B[GeUdAUOD 
snjd uodey op Jop[reaes] JUDANEd sINaMITAIOJUI Sat aonb gIJUOU! JUO SoINaQ\Ue SoYydIOYOEI saq ‘OVI JUSWOUUOIIAUS UN 
SUPP SINOMITAIOIUL SAP JUSUIDIOdWOD J] INs OUANTUI oUN JUO sosTeUUOT|saNb sap UoNdadU0; FT Jo suvIDG Sop UONdadU0D ET 


aWwnsaa 


(SHON AAAVN 


UIV1LI9} I[ ANS IQUIU UOHVSTN .pP 9ILIVJ 9p 9dUdLI9dx9 ouN ,p s}vI[NSIY 
OV] 2[Npou un suep suoysanb sap a1p.10 | 39 SUB.ID9 SOP UOIWdI9U09 BT 


See 


epeuey enbijsiels 
vee-S7c ‘dd ‘Zz .U ‘92 ‘OA 
000zZ eiqwacep ‘ajenbua,p senbiuyoe | 


“suoTjeor[qng o3e¢ 
SUOpUuO] “3ulapou jarajijjnu paouvapn pun I1SDq 0] uoYoNpo.4ui 
up ‘sis{joup jaaajujny (6661) “A “AANSO" 1° “L ‘SYACLINS 
‘9TP-SOP ‘OE ‘AuDNG » AujONG ‘Jepour JeauT] [PoTyoIeJaTY 
‘OCZ-T6T FOUND OP JEM HOA MON - oy} SuIsn erep [eUIpmyIsuo] Jo siskjeuy “(9661) “\L ‘SYACLINS 
ulipiog ‘(aysoutoy “¢ 19 ‘Jasuq “Q ‘sp_) ‘sisdjoup vIvG JauDvg ul 
sanbiuysa], paouvapy ‘asuvy) fo sisApoup sueq ‘elep jeulpny suo] 
JO} S[PpOU JSAs[HIN (9661) ‘H ‘NIZALSATOO 1° “W “ONVA 


“ssegq-Aassof :oostouely 

ueg ‘suolsanG) Bujdamsup (S661) ‘S ‘NVNGNS 9 “N ‘ZYVMHOS 
“SSIg DMWMApKoY :YOX MAN] “7xajUo0D 
‘QUIPTY :oseoIyD ‘sdaaung pup SuIpsoy{ ‘W40.J UO1ISANG UO Sjuausadxy :skaaing apniupy 

ul spaffy asuodsay ‘(pL61) ‘N ‘NUNddVvUd ? “Ss ‘NVIWGNS ul Sdamsup pup suoysan© (1861) ‘S ‘YASSAad 1 “H ‘NVWNHOS 


joued sed sajanbus sep seo 9] SUBP INaMAeIANAJUI,P SJaJJo SAP UOHESI|SPOW :}PjeAsoo7 38 AVexd!q zz 


“St]I9 A -JaSULIdS :yIOX MON “sUulduvg Aaaing pajsissy 
12POW (7661) ‘f ‘“NVILLOYAM 19 “dd “NOSSNAMS “DO “TVANYYVS 


“ore-1e 
‘Q] ‘sousiiojg jouolvonpy fo Jouanor ‘yoreasel JeUIPNyIsuo] 
pue [euonses-ssoio ur suoneorydde yyM eyep poouryequn 
JO} [OPO sjoaJJo WopueI pessold V ‘(€661) ‘MS ‘HSNANAGNVa 


“UOpUOT Jo AjISIOAIUG, ‘UOHONpY jo aIMNsu] 
‘yoofoig S[Opo| [PAI[H[NJ :uOpuoyT “eQ'] UOISIOA “aouasafay 
punuuoy) uTW (9661) ‘D ‘“ASNOHGOOM  “f ‘HSVaSVu 


‘OSE-LEE ‘6I ‘SausDIS JoUOlDONpT fo joUdNOL ‘[EPOUL [EAeTH| NU 
8 BUISN SOINjONIJS WOPULL PdIJIsse[9-SSOJO PUe [eOTYOIeIOIY PoxTul 
jo sisdyeue wolsyjq “(y661) “H ‘NIDLSATOO 1° “f ‘HSVASVU 


‘TCOI-LIOL ‘YoNDio0ssp [DINSIDIS UDIdaUpy 
‘spoyjayy yodvasay Aaaing uo uolday ay, fo séulpaar0dg 
C6. - 16, Apmis Joueg uonoo[q uelsjagq oy) ul asuodsozuoUu 
jo woneneaq (L661) ‘WV ‘NOLYVO “OD “LATHASOOT 
“QET-EIT ‘¢ “ABojoyodsg aampusoy 
payddp ‘sKoains ul sainseaul apne Jo spueulap sAnTUsOO 
9y) YIM Butdoo soy sarsayes esuodsoy (1661) ‘Wf “MOINSOUN 
‘suoneolqng a8eg :uopuoy7] ‘suyapopy 
jaaalujnyy sujonpo.yuy (8661) ‘f ‘MNAAT AC 2° “OT “LAT 


‘OdS] :uaAngy] ‘azipuuoljsanG puv 
yooqapo) wnisjag Apnig uoyj9a,y [042UaD 166] (S661) dOld/OdSI 


‘19b-6EP ‘ASTM 3JOK MON “(UBUIPNS “¢ 19 
‘ZJOMOIUIR|Y “W'N ‘BJOQAT “qT ‘SOAOID "'Y ‘IOwWTg ‘d'd SPq) 
"SKAAANG Ul SAOAAT JUBUAANSDAPY SUC] “JOPOU JOAITH[NW ke "eIep 
Addins Jo Ayyenb ay} uo sonsuajoeseyo yuapuodsal pue JOMeTAIOIUI 


Jo 1aIJ9 OU (1661) ‘Dl “LHAM 3° “ad ‘MNT A “ff “XOH 
“81€-00€ 

‘TZ ‘Yyodvasay puv spoyjayy jv2180j01905 *s\a}Jo Juapuodsas 

PUL JOMAIAIOJUI JOJ S[OPOUI UOISSaIZEI [ROTYOIeIOIH ‘(P661) ff XOH 


“SSO-SE9 “ES “UONDIIOSSP [DINSIIDIG UDII4aUuLy 
ayy fo jpusnor ‘sy{nso1 KoAIns JO Aovinooe ay) UO JOMOIAIOJUL 
ay} Jo souanyul “(8c6l) ‘Sd “SHUVW ® “HU ‘NOSNVH 


“KOT 
‘YJOK MON *S1S0D Aaaung pun 40.117 Kaaang ‘(6861) ‘WY ‘SHAOUD 


‘ploury plempy 
:UOPUOT ‘sjapoyy J0ISIDIS JaaaiujnW (S661) ‘H ‘NIDALSGTOD 
(ES tal CS 
‘97 ‘Asojo1009 fo maiaay Jonuup ‘souR\sqns pue spouyie 
‘S[OPOUL [OASTHIN “(p661) “A'f “TVLSTAYOd 3° “WL ‘ALDYdIG 
‘OdS] ANNINsuIsysozIOpu_, 
YOSIBOTOIN0S :UdANIT] ‘UuoljIalq jp4auay 1661 
ay] YIM UOIaUUOD Ul ApNjG ,S1ajo4 ay] fo yOOg a24nog (F661) 
“U NALYA 1° “f “LEITIIA “IW SMNOGADNAMS “Vv ‘NOLYVO 
‘ase :UOpUucT - 
yleg Ainqmay ‘spoyjapy sisAjoup vjog puv suoyvoyddpy sjapoy 
ADAUIT [DI1Y4I4DAAIFH] (7661) “S ‘HSQAANAAGNVY 3° “S'V “MAU 
‘UDANIT (1) Y SISOTOINOS 
juowouedeq/OdS] :ueAnay ‘auinuuoisanG puv yooqapoD 
‘wunisjag-S4apupj.y Apnig uoljoayy jo4aueH C661 (L661) 
‘W‘MNOGADNAMS 19 “V ‘NOLYVO “f “LEITII “(NEL 


ce 


LV ELS 
‘pl ‘Yauvasay suyaynpy fo JOUANOL ‘BdUCLILA PUL SPIG JOMSTAIOIUL 


Jo soinseaJ] “(LL61) “f ‘“SNHSAALS 2 “'T ‘AATIVE “A “UV TIVE 


aIHd Veo Tala 


“gjorye yuasgid np 
aIped a] DUOp assedep 30 oddel un,p Jafqo | ‘tos ua ‘yTeIOJ 
uoNRodx9 INI] “SdANOI SITQeUeA Op S}JOUNSIP soTquussud Pp 
UOTIV9IO SOAR JUSIEIJIp oseoWeIed UN JUdIISsad9U SaTepoul 
XNep Se ‘sIojajno], “ajepowl a0 yWOALIDap (QOT-6S|1 “d 
‘6661) Jaysog 10 siopfiug yuepuodel ne jo INIMOIA 
-I9)UL | @ SOI] SNPISSI So] BUS UONIEIIOS ouN jeJouseNUS 
Inb ayeuolsel UONRIIVA eT INOd SUID) UN eINOUI sJepoul 
aD ‘sjuepuodel sop Jo SINOMOIAIOWWI Sop XNevoAIU xne 
(uorseal e]) sreyuowgtddns neoatu un Joynole & WeI9}SISUOD 
UOTINTOS ane sUP) ‘SNPIsel soy NUS s]JENIUSAg UONPR] 
-91109 dUN Pp 9dUd)SIXO,| OP SOpl ouN DUUOP UOT}OeIOJUT .P 
SULIO) NV DORIS B[QPOU Np UOTJLIOT[OW | JOUTLIDJ9p op ona 
Ud 3S9} UN Pp UONNIEXa ‘J ‘a[Qpoul ap a1ulas 39 esodold (6[ | 
‘S66 1) UlaIspjoyH ‘sarloyeg[e sjajjo xnap soy aUS UOTN}ORIOIUT 
oun,p ajduios jueua) oxajdwioo snd ajapow un sajsnf{e 
B UWLIISISUOD UOTINIOS dINI|[IOU SUP) “SINOMITAIOJUI xneV 
Sdl] SNPIS9I So] sJUBPUOdaI xne JONqUe & d}SISUOD dII[9 Jed 
‘ouejreduit yso ¢ anbrydess ne agjuaseid UONeOIJUIOA BT 

‘IRIA SO 
asiaAut | onb sasoddns op snjd uou sed jouned ou onbrydes3 
Y] ‘SPAgTP snjd jUOS SINIMOIAIOJUI XNV SOI] SNPISaI soy Is 
so[qrey snjd ong Be souepud) JUoUTeIA sed UO ,U sJUepuodal 
xne Sal] SNpIsel soy “Uepusdeg ‘anbiydess np syones 
9109 np onb a}101p 9309 np 2s] qIej snjd sa suOTFeAIOSgO .p 
SIQUIOU J] ‘SINOMOTAIOIUI XNE SII] SOUIQIIXE SINsTVA Sop 
esneo Y ‘alatnonsed gyejn3ei sunone sArasqo .U UO ‘ed 
B SOSTU SUONLIASP SOD “SINOMATAIOIUL XN SagI] SoUIgIXe 
SINZ[VA SIT INS 39 Ney I SIOA syede.p sIquIoU pues 
snjd 9] Ins uonUS}e | oe onbrydess 99 ‘nesAnou 39q 


“SINIMOTAIOIUL XN S9T] SESIPJepue}s snpIsel 
So] UOTas sJUepuodgl xne sal] SasIprepueys snpisoy *¢ onbiydeasy 


SunamelA/e}u) XNe Sel] SNPISE, 
00 co 


= 
Ss 
a 
£ 
i) 
3 
rs 
® 
c 
x 
of 
° 
3 
a 
® 
3 
a 


0002 aiqwes~ap ‘ajenbusa,p senbiuyoe | 


"¢ onbrydess 9] onsnqi.nb 99 sa, 
"SINIMIIAIOIUL XN SOI] SNPIS9I So] sJUepuodal xne JueNqLe 
ue ssgujodAy 31199 JUSUIOJDAIIPUI IJOIJIIA ap a[qissod ys 
[I ‘SINIMOTAIOIUI xNw S9IT ¢LZ 19 SJuRpUOda xne ser] snpIs 
-91 8Z0 € JUSTIGO UC ‘sed JuspuodsadIO. ou s9}IUN sad anb 3a 
satjoodsal g}1UN INI] & SI] UNOeYD JWOs snpIsal set onb sored 
‘IOJLIDA B OTIOLJIp snd yso asgujodAy 9999 ‘JuauaTFoINwNY 
"Sg[aLI09 og sed jUoTeIAOp ou jUepUuodsl Ne 30 INIMOIA 
-I9]UI | & SPIT SNpIssI sa] ‘onboidio91 souepusdeput Ing] & 
ye1} @ Inb oI[99 so JURpUOdaI Ne 39 INOMOTAIOJUL | B S9TOOSsSE 
sollojege sjojjo soy yuRUIIDUOD ssayjodAYy anne TJ 
"$Je][NSQI Soy Ins JueyOduIT Jao 
un UO souIgNXx9 sIne[va soy onb sAnoid sunone yUsUUOp 
OU S}e}[NSOI SOT STRUT ‘SOUIQINXO SINe]eA XNep soap uUOIsNID 
-X9,[ & DORIS ‘JUUIOIQSI] QNUILIIP & INOMOTAIOIUL,| & IPI] 
OURLIPA BT “¢ Nes[qe} ne d sJapour 9] Inod sgjuaseid xnvo 
B SO[Qe[qUIOS J10J JUOS NeI{qQe} 99 SUP JUSINSIJ INb syuarory 
-J909 SY] “A9AgT9 Snid e] BJONpIses Inoyea ve] e puodsai09 
mnb tnpao 30 afqrej snd e] a[jonprsoi inoyea eT Be puod 
-S91109 INb IN[9d IIOARS & ‘SINOMIIAIOJUL XNOP NOXS SUOAL 
snou ‘p neajqe} ne sajuasgid asdyeur | & Jopgooid nog 


T0S897 TI? 
(184‘0) O@r‘El *o 
SOINSOU! SOP NIAIN 
(0€S‘0) 601‘L PS 
sjuepuoday 
(€€€O) S6r'7 ane 
SINOMITAIONU] 
Z NedAIN 
aIlOesTV 
(ZOl‘O) LEV zassaid 
(060‘0) O9I‘T [essoid 
(6v1‘0) 626'I- 91LB]O9s op NedATU 
(€ST‘0) 078'T axes 
sjuepuodgi sap nesAIN 
(Z91‘O) Esse a]uUe)sUOd 


SOINSOUL SOP NVIATN 
oxty 
(sasgujuored oud *}-"9) 


SINIMOIAIOJUL XN SO9I] SOWIQIIX9 
SINLA SOT SUS S9QUUOP op I[quIASUD | op osATRUY 


p neaqe 


‘p Nes[qe} nv yeJ suOAR Snou nb 99 js9,-) “SOUIgIX9 SINBTBA 
So] SUBS SAQUUOPp ap s[QUIVsSUS | JosATeUR JUSUTUApIAg Jnod 
UO SIe “(67 “d S66 Ulalspjoy) xnNesAIUN[NUL sefgpoul sap 
SBD 9] SUP SUONPOIJLIDA SOO oulej INOd sadedIjJo Sopoyysul 
op sed asodsip ou uo ‘ayjanjoe ainoy,.| YW “INOMIIAIOIUL | & 
sJueyal snpisau soy inod soutanxe sina[eA sap 19 INeY dI SIOA 
sosnoiquiou snd suonetAgp Sep UOSTeI P| JOUTULIOJ9p Inod 
epnig,| aAInsinod op afNn ang wWeunod [I stews ‘ajsojtueul 
alfewioue sunone jUa}UasSeId OU SNPISEI So] :SoJUBAINS SO] 
yuos sonbrydeis sao ap Jon ned uo | anb suorsn~ouos sa] 


“Q[RULIOU IO] BP] B BOUaTBAINbY ,p 9109s 9] 
UO[AS SINSMIIAIOIUL XNK SOT] S9sIpJepuLys snpisay *Z enbiydesy 


PUIOUIOU Op Ise} NP Sal09s 


SPS/Psepuazs snpises 


‘oayones 
INsLgJUI ULIPed os] suep onb yop Inouedns uvIped 92] 
suep SUOTIRAIOSGO ,p BIQUIOU pueIs snd UN Issne IN] JUeNUOD 
onbrydeiz 9] ‘WoIOINs oq ‘SINOMOIAJOIUI xNe Sal] SNPISSI 
SoINe SOP BdYDINOJ B] Op SJOYap-Ud IJaquIO} JUaTquIS 
inb ‘j101p Inotigdns uelIped a] suep soentiIs sa][a0 JUSSI 
-gynored ‘uonusye | JUSINIe SouIgIXe sINe]eA soureLIIO 
‘anbiydeiz 99 suep ‘slojajnoy, afeuoseip ey] e wodde sed 
jou e909 .p snjd uou sed sasosqo .u uo ‘Zz onbiydei3 9] Ins 
“oYyoNes INILQJUI ULIPeD a] suep onb jOIp Inatedns 
UBIPed J] SUP 9AgTP SNTd 3Sso SUOTJBAIOSQO .p eIquIOU a] onb 
JOUZI[NOS SP JUSIAUOD [I ‘SINIT][Ie Jeg “s[eULIOU IO] eT op UON 
-R[OIA OUNINL JOIZJUI ,pP JUS}eULIOd ou Ja SOIT] ZOSSe JUOS 
geuoseip ey] & woddes sed syreo9 soa] ‘anbrydeis 90 sueq 


Q[eULIOU ToT BP] e QOUITeAINbY;p o109S 
3] UOTas sJUepuOdal xne sor] spsIprepue}s snpIsoy *[ anbrydesy 


9/8 UOU ep jsej] NP seasons 


Sgsipsepusjs snpises 


"SINOMIIAIOUL XNV SOT] 
sgsIplepueys snpisal sa] Inod ‘7 onbrydess 9] 39 syuepuodgi 
xne SgI] SPSIPIepuL}S SNPIseI Sof Inod snusd}qo oqInod v| 
ajuasaid | onbiydeis a] ‘snptsei soy Inod safeuLou s9yI]1q 
-eqoid sap onbrydeis 9] jUeUTUeXd US S[PULIOU IO] BT JUSAINS 
INOMPIAIOIUT J Inod 19 yuepuodai gj Inod sanuajqo *”"*"o 
ap singe say a[anbey uoyas asayjodAy J Jonfeag yned UC 
“INOMOIAIJOUL | B 39 JUBPUOdAI Ne SaIDOSSE SaIIOJVYTe SIOJJO 
xne yer) yuo sajueyodut snjd so] sosgujodAy soy “Jorutap 
J] INO *S9sI[NN safepour sIOAIp soy juetndde s sajjenbsgj Ins 
sasatodcy sa] SUOALID9p snoUu ‘UOTes agLeNb e] Y 


f AXUNNV 


jeued sed sajyanbua sap seo a] SUP INEMAIAIAJU!,P S}AJo SEP UOHES!|9POY\ -}Pj9eASOO07 jo Maxold GGG 


"SOUIQUI S93] S99}S91 JUOS SasUOdgI ap SotIOS9Ied Sa] 

"«{ XNVUINOl So] SUBP NO UOISIAYT9) PT B ‘OIpes 

RI @ sonbrytjod sayjaanou say] snoA-zaAIns souanbej otjanb 
Y» :onueaep jso uonsenb ey ‘a[dh9 ougIxnep Np S10] 


‘srewel=¢ ‘uoWorel=p 
‘sduia) ua sdura) op=¢ “uaAnos=z ‘sinofno) (anbsaid)=] 


:SOJUBAINS Sd] JUSTEIS 
sasuodgl ap sati0saie9 sa] «{xneuinof say suep sonbrtjod 
SO][PANOU Se] sNoA-zast] sdUaNbesy at[anb Y» :31e19 assoid 
B] SUP TAINS 9] JURUIEDUOD UONsenb eT ‘a[DXo satwIaId 9] 
Inog ‘aygnbua | ap safoAo xnap sa] Inod swigui vy sed 71819 ,U 
essoid ve] suep onbiyyjod ey ep tains 9] Ins uoNsonb ey] 


enbyoyjes uou 01 6 8 LIS bE ZI O OnbioyED 


OBO BP] DOAY «9109 op a1e9 
Re] yUoWTdulls Zzoyour ‘«anbiyoyyed uoU» no «anbijoyjed» 
3so ted un ainsoul ajjanb suep sed Zaavs dU SNOA IS “ased 
OUIQUI B][ INS syed soy ZooeId ‘«sanbiyoyyes UOU» Issne no 
«sonbrjoyyed» Issne uos sted sinatsn{d no xnoap ‘snoa uoTas 
‘Ig ‘«anbijouyed uOU» no «anbijoy}ed» Js9 UONseNnb usa ned 
3] aJenbe] suep oinsow ey e xno a] puodsao0s nb QZ 
OIQUINU 911e9 BI Op ased P| Ins ted onbeyo ap syed PY] JaoR|d 
Zal[INeA “«sanbrjoyyed uoU» no «sanbrjoyyed» Juos senbrtyod 
sted sot onb Wp uC» :ajURAINS ke] We}9 UONSONb eT 


T GXdNNV 


"Q[OIUe | Op DINILI9JUe UOISIOA OUN JURUIIO 
-UOD SJNOMSUOD SdaITeJUSWIUIOD Jo SUOT}SessNS Ino] Inod 
SINDJVUTWULXS S93] SUOIDIOUIOI SNOU ‘UIJUA “UTY [P1130] np 
UOTLWO[AXa | JURUIDOUOD S9TIIN WO} SaITeJUSWIUIOD suTeyI9O 
Inod (uopuo’T jo AjIsIoAlUA ‘UONRONpY Jo aynjWsuT) yseqsey 
UOL JUOWITeS9 SUOTDIOWIII SNON “1dI sagyuaseid suotye}o1d 
-I9)UI Sap no saskyeue sop sayqesuodsal yUOS dU BYdIBYIeI Op 
9U99 9] IU Uedep ne saguUOP S9zq I][IendeI JUO Inb souuOsied 
S32] IN ‘“Seyamnynd 39 senbruyse) ‘sonbijnuaros sairejje 
Sop XNeIQP9} SBDIAIOS so] Jed goueUl} Iso GOId-OdSI1.1 
‘QpueUIe]J UOIS9I L] INs sofeUIZLIO SaguUOPp Sz] I]][IandeI UO 
UsLIS0g pUL[sOY Jo UOURD UUY ‘MnopesUAMS oeP] ‘ONIIG 
sonboer ‘saguuop so] IWINOJ JOAe snou ap qOId-OdSI 
SOTLIOJIO[9 SOYDIOYIOI OP BUDD |] SUOIOIOUIOI SNON 


SLINGNGIYaNAa 


‘joued sed aygnbua | ap sajaA90 say sno} ap ayutofuoo 
osdjeur | op jueyodut asejueae one un ‘ajuepusdeap a]qQeLI 
-RA PB] OP J[INIUSAD UOTNJOAY ouUN JasI[gpow op sed jusren 
-Jouliod ou saskjeue sao ‘stojaynoy, ‘[oued sed ajygnbua,| ap 
9[9A9 anbeyd Juowsiedgs yresAyeue UO | IS SUOT}OBIAIUI S90 
SINSUI WeLINOd UGC “INSMIIAIOIUI 39 JURPUOdaI aNUS SUOTIOR 
-I9\UI Soy siNjoul A.p gyt[Iqtssodut | asuaduioo anb snjd 
Q[QpoUul 99 ap assoydnos vy ‘asAyTeue aNoU aNUOU 9] SUTWIOD 


Lo? 


‘Juonbuew soinsoul sinoisn{d no oun sjenbsay Inod sjuep 
-uodal say 19118.) Op Issne youIed xnesAlUN[NW sJepowl JT] 
“aSATRUL SUIQU PT JUSUIDJDeXA JTBIO}ISsaogU B[DAD onbeyd op 
SIO] SINSIMOIIAIOUL XNVIANOU Jp 93ND UO J NO 39 SINOMIIA 
-IQJUI SUILIIIO JUITIOI UO] NO ‘aseJURARP SUIQUI NO ‘saTdA9 
anenb no sto yue}dui09 jaued sed ajgnbus sup ‘ayuowIsNne 
SdINSOW Op BIQUIOU JI IS JUSWIATV39 JUSIAUOD dgSIOIO 
UONLOIJISSE[D & B[Qpow aq] ‘afdnos yo} 39 BIN WoOF [MNO 
un ang weunod sjapour 90 anb ayuoul asAyeur | “uepuseded 
“INOMOTAIOUL,.| 39 JURpUOdsI 9s] JURSLIQ}ORIeD soTqelIeA 
SI] DIVUS SdT[9O Tu ‘asATeUe dIQTWIOId BI AP SIO] WJ SUOAR | 
SNOU SUMUOD “INIMIIAIOWUL.| B IPI] BOURLIVA ke] 39 JULP 
-uodgi 9] JUaSTI9}OeIeO INb saTqelIeA So] aUS SUOT}DeIOIUT 
S93] Jasl[apoul ap sed jounsed ou [I] “Soy sos Issne In] eB 
JQpow SWIXNap JI ‘sIOJaINO], ‘UONeUIOJUI Pp oped 31939] 
oun.p xud ne ‘asinbor allow9sw ap syquenb ey] Jonurwip 
ap JouLlad 3a Juessind zasse 3so UTPy [SIdI3O] VY “owWsOUP 
JusUN}Ie] op gyoedeo oun a8ixo mnb stew ‘gtidoidde 
[QNo un 4sa (ashTeUe SWVIXNEp) s[QpoU SWIQIXNEp I] 
‘joued sed ajgnbus 
OUN Pp SaToA9O SIOAIP INOd JuepuOdal OUIQUI Ne INSMOIIAIOUI 
OUIQUI J] J9}90JJe.P ‘a[qissod sinofno} tu ‘ueINOd Is sed 
}S9,U [J] “Seo op aiquiou yhed un e nb ajdepes ou (askTeue 
algnueid) sjapow Jormeid ay ‘gudoidde xneoarunfnu 
Z[Qpoul a] JueTJIO9ds ud saxodulod zasse aBepuos op surid 
ap jJuRyNsei sanbijduios ainjonys eB saguUOp soa] JosATeuR 
yned uo, aonb snuoul 101 sgjuasgid opnje,T ‘Joued sed 
sojygnbus sap sped 9] SUPP INOMOIAIOJUL Pp 39JJO,] JosATeue 
Inod sajgpoul sIsAIP op uOHesI[NN,| sp aJdwoo jusuusy 
suOIsnjouOS sal ‘anbizojopoyjau ona op julod nq 
‘Q[QIeJ ISO 9}LIe[ODS Op NVIAIU J] JUOP s}uUepUode 
so] Inod ayo} snd 48d INIMIIAIOUI.| & II] OURTICA 
eR] onb ayUou! asAyeue sIgIWAId P| 39 9AdTP SNId jsa gyLR[OOS 
op nedalu 9] yWOp xnad aonb ajgnbua,] op JoMje1 os op 
sa[qndaosns snjd yuos s}inysut suloul say syuepuodal soy onb 
juaNUOU ([TZOI *d ‘266 1) UOURD 19 IP[eAsooT ‘gUINSd-snos 
nod anbyenb qos “InaMeIAIOIUL,p [eQo]s 12JJ9,] “Uenbgsuod 
Jed ‘a asATeue SUIQIXNep P| Op adesap as Inb INdMdTAIOJUI.P 
JOJJO,] onb 99 e sipuaye,s WeLNOd UO ‘UOLID 19 IP[aASOO'T 
op soe] op suorlsnjouod sa] 39 osATeue sIgMUIOId ke] 
OP SIVI[NSaI SO] QUUOP JULI “INIMIIAIOJUL,P J9JJO,] JUBUIO 
-U0D PUOJ OP SUOISN[DUOS So] JUSOUENTJUT suONIeIDIUT $29 
anb afqeqoid nad stojajno} 3S9 [J] “INIMIIAIOUL Pp JOJJ2,[ OAL 
Ji3e19}UI JUSANed Juepuodgi np sanbysugjoeIe9 say ‘askTeuR 
QIQIUIDId BI SIJUOU 9] OWIWIOD ‘IVD INIMOIAIOJUL .P JOJJO,] Op 
INopuLsS P| INS JaJJo UN je VJOAD OUIAIXNEp Np SIOT SANOET9S 
asuodgs-uou ey] onb snjd uou afqeqoiduit sed sau [] “«sed 
Wes ON» osuodal ke] ap uOTesI[NN,| op s[e1gues UOTN[OAY 
ounone siojo}no} juonbipur,u sasATeue sou stew ‘ayuepuadeap 
J[QUIILA BP] SP UOTIN[OAY,] BUIIDUOD INb 99 Ud s}eI[NSOI SOP 
SUTIqesife1gUgs L] WUT] JNOe]9s UOpuRGe 19D “(L661 UOUED 
32 IP[easooy) sanbrydessowgporoos safqeuea sonbjeanb 
ap 39 onbniyod ev] nod yg19}uI UOSs op “Uepuodsl Np USUI 
-980] op saiyepoul sop puadap aq “osloyegTe JUSWIsIETIUS 
sed jso,u aygnbus | op a[9Ad awWatxnap np so] asuodeJ-uOU 
eI ‘jaued sed saygnbuo sop yednid ey] suep owui0d 


000Z eiqweassp ‘aj}enbue,p senbiuyoo | 


“«sed 11¥s IN» Sasuodai soz J9}1e.) op 39 SaTIOLJIp suonsonb 
Sg] Josod op uodey ke] Ins suonoOnNsut sap suduios 4K 
‘SINOMAIAIOUL XN UOTULIOJ oUN JoUUOPp ne} [I,nb onbipur 
39 Juepuodg1 np ssuodg1 op snssoooid np juoworduts 
sed oy[nsgi ou «sed yes oN» osuodal ve] ap x1oyo a] onb 
DANOId INSMIIAIOJUI .P 19JJ9'J “IQNbua | op Ne, | e a[DAO 
un.p «sed Wes aN» ssuodal ke] B SINODaI NP sATROTIIUSIS 
UONNOAY sUNONL sUOJeISUOD OU SNON “JULYOdUIT INaMOIA 
-I9}UI ,P JoJJo UN Pp JUDUSIOUI) SasATeUe SOU SaINOT, “INIMITA 
-I9JUI | UOTAS JUSUII[qe[qUIOSIVIA OLIVA XIOYO 92] ‘WoOIOINS 
aq ‘jolns 9] Inod ygigjur uos eB nb Isure “uepuodgi np oxes 
NV 39 9ILIB[OOS Op NedAIU Nv 9IT ISO BsuOdaI 9}}90 ap xIOYO 
YJ] ‘«sed yes oN» asuodal ve] op uOnestNN | JUeUIODUOD 
SOINILIQIUL SOPNII,P $JeI[NSOI $9] DIOQOLIOD asATeUL ONION 
‘puoj 9] Ins onb Isure ats0;opoyyeur 
RB] Ins jus}yIod opnj9,] op sayelgues suoIsnjoUOD sa] 


NOISSNOSICG LO NOISNTONOD ‘8 


“ol qrey 
yso onbryijod ey inod ygsgjur,, anb gaoje snjd jueine,p iso 
«sed I@s ON» Sasuoda op o1quiou a “«sed j1es aN» osuodaI 
R] Op uoNesi[NN.| op Inojorpeid un UsWaTese 3sa ossoid 
RI suep sonbyiyod sajjeanou say JUeAIns sjuepuodel so] 
g]jonbe] suep sinsoul eT “QAI[P9 ISO aILIeTOOS op NedAIU 9 IS 
g[qIej snjd ysa sasuodal sao ap o1quiou 9] Jo SouWOY say onb 
JUSANOS snd «sed jes AN» osuodai eT & JUSINODEI SOUTWIOy 
sa] ‘oshyeue oigmueld e] nod aonb owigui ev] Iso sanauiesed 
sop uonejgidiajur,.| ‘puoj np yso Inb 90 Inog ‘syuapeogid 
so] onb gisnfe xno jUdUIA}}0U Sd B[QPOU 9d 39 JOT 
-IUBIS JJJO UN $9JNO} UO SaT]q ‘sJUBpUOodal so] JUBSLIDJORIeO 
‘STqeleA Saj jUaTUOD Inb sfepour J] 3sd 9 B[Qpow! oT 

*sduray 
NP SINOd ne IATJVdIJIUSIS UOSeJ ap sed onjoAg.uU «sed jes 
ON» sasuodal ap oiquiou 9] ‘ayquiasua | suep ‘anb ainjouos 
ouop suoAnod snon ‘19-In[90 ap yuouIaysnile | sed aloraure ,U 
Q[Qpouwl a] suep ynole uos Jo JreoIsIUsIS sed jso,U s[qQULIeA 
91199 op 19JJ9,| ‘NeaAnou 9q “(GANNY) Sduei op sfqeiea 
B] SNOUT SUOAR Snou ‘(q s[Qpour) JURAINS s[Qpou 9] sue 

"SINIMOTAIOIUL 9.UD Nb ay0j syd 
JUDUI9}}0U Iso sJUBPUOdaI aIUD sITTIGeUeA | onb suoje}sUOD 
snou ‘snjd op sioj oun ‘sIeul ‘sINOMOIAIOJUI OU UOTRLIVA 
ey onb apueis snjd stoj burs sa syuepuodes oUS UOTIVIIVA 
RB] onb altp ap yOaLI09 je B yNo} sed 3sa,U [] “(ST I-LII “d 
‘S66I UleIspjoH) ane, onb soyun,p puels suloul JUOUI 
-ayJ0U aIquioU UN a}duI0D UOTBOTJISse[D ouN ,NbsJO| ‘ssouLLI 
-8A Sap dAIeIOI souRWOduI,, souspnid dade Jo}gIdIO}UI 
INV] [I ‘SUIOWIULAN “INOMIIAIOIUI,P JoJJo uN oJsSIXo [I,nb 
dANdId e] NVIANOU Op QUUOP SNOU I[QPOU 9d ‘OUD “PATIL 
-IUBIS ]SO9 INOMOIAIOJUL | B III] DOULLIVA BT “INIMIIAIJOUL | 
& dal] sJUesodwios oun jo JUepuodgl ne ogi] aJUesoduI0D 
aun ‘saInsau xne sgt] 9JUesOodu09 oun puaiduios ajuepued 
-9P [qQeLIeA ke] Op DoULLIRA kB] STeUl ‘SoATVOI[dxa saTquLiea 
ap sed juaUOD OU [I :[NU s[QPOW! Z| 3SO B B[epOU JT 


‘as[VUR | SUBP NRIATU OUIZISION) UN sed yUd}UAS 
-91dai OU SINOMOIAIOUL Sot onb yuoWoITe[D Janbipur inod 
‘syUepUOdgI Sz] 39 SINOMATAIQIUL Sa] BqoTSue Inb ‘Z neaAtU 2] 
910U SUOAR Snou ‘arO}egTe oIed Re] Y ‘aUeyOduIT soUdIaTJIP 
oun ajuasgud sreur ‘7 nea[qei ne WO} s]quIassar neve{qei aD 


B® O[QPOU! Ne JUSUIOANVIedUIOD y :2ION 


JPV 
U'7rOL7Z €OILLZ ULILLZ TW17- 
(LZS‘0) LIO'L (SE9‘0) OO8'IT (SE9°0) OT8'TI hyenas 
sjuepuoday 
(€9€'0) py8'Z (89E'0) 9IL'Z (ELE) LLL‘Z Lesgees 


SINIMOIAIO}U] 


Z NOAIN 

amloyespy 
(ZOL‘O) L6I‘I zassaid 
(060‘0) S8I‘T [assoid 
(8r1‘0) r16'I- 9}Le]OOs 
(€ST‘0) 808°T axas 


sjuepuodgs sop nesAIN 
($S0‘0) €S0'0- 
(S910) p98°E (SST‘O) L96‘E 


oguue 
(9ET‘0) 768° 9]ue}suO0d 
SOINSOUI SOP NEIATN 
oxld 
2 gJQpoul 


q 2[gpour 8 9[Qpour 


(sosgyjuored a3ue *}-"9) 
sjuepuodgi sop ayquiasua | ashTeuy 
¢ nvslqel 


"€ nvaqe} ne sajuasgid yuUOs osATeUR | Op s}eI[NSaI SAT 
‘juouapides JUSWUUeSIJJNS YBIOAUOD & B[Qpou 
a] ‘uodey onN100 op anbiddy ‘sapni[20 pp snjd ne ‘guuop 
B SINIMOIIAIONU! Sap Jo sJURPUOda Sap dgSIOID UONRIIJISse[D 
e] sfanbsa] Inod sajquiasue-snos jdes onb guuop eu inp 
-9001d v'T “9100 op S9SSIP] 1S9,U (SINSMOIAIOIU TU sjURpUodgI 
Iu) InoLadns nesalu op gyUN oUNINY “SINOMOIIAIOIUL LZ 
ed samaIAJojUl sJUBpUodgl OZ € INS SoINSOU /6¢ p IWeISI 
Snou [I ‘YdeqS Kg spuewuuod vy Jed sognbipul suoneAsrosqo 
SOP UOISSIUIO SQIdY “SINIMITAIOIU! ¢/Z 19 SJURPUOdSI QZ € 
‘SoINSOU Q6L p SUOTAR SnoU ‘yedap Ne :soguUOp sop UOTS 
-soiddns ve] WNUITUTU ne JURSINpel US o_eWITxeW UOTRIedgs 
RB] alInpold eB yURSIA agIOI[QUIe aInpgd0id oun soURT, INb 
youeqsx { SpuewuUod Lf gst]IN SUOAR SNON “(€6-68 “9661 
asnoypoorA 39 Ysegsey) UoNeIedgs 99199 B Jopgo0id Inod 
sonduo0s (YeAS KX 12 YOIVASX sopuewulos xne 99p13) 

somnpgo0id saUTeyad ajJO NIMTW/UTW [P1180] 37 
‘aovorgo sntd 
uonviedgs ev] orpuas nod (sreuref yuswionbyeid aarasqo .U 
uO.| onb sINsMOIAIOIUI Sop B 19 SJURpUOdg! Sap & SoINsaU op 
SUOSIVUIQUIOD) SUOTRAIOSGO SOUTELAD BINJOWO ,pP a[IIN 3SO II 
‘SIOLIeg “SINIMOIAIOIUI Q] 12 SJUBPLOdgI (YC UOTes sagIsIssE[D 
SOINSIUI CQ ep sednois Q] JURUSJUOS seguUOp ap a[quIesua 
un,p osAyeue,]| onb s[nojeo op snjd apuewiap sinemotAsoqul 


jaued sed seyenbue sap seo a] SUEP INEMAIAJAIUL,P SJOYO SEP UOESI|BPOW “IP|9ASOOT 38 A1B}491d 02z 


OOL 18 sjuepuodsl QOS UOTAaS SadIJIsse[D soINsOW COO I 
ap ednoiz [nos un,p asdyeue TJ ‘SINOMIIAIOIUL,p Jo sJUepuod 
-91 dP SIQUIOU PULIS SUTOU UN UOTAS SodIJISse[O ‘SoINsoUl 
ap SIOUNSIP Sadnois sap 9ydIOYIOI SUOAR SNOU ‘1dJ *soTN][eo 
OP 9AI[P SUTOUW «JUSW9}0U» sIqUIOU UN yINpoOId dgSIOIO 
UONLIIJISSe]O eB] Sfanbsay Inod sajquiesue-snos usa soguUOp 
op s[quiosua | JuRIedgs US B[QPOU NP UONeUINSO Pp OSSajIA 
P] JOIONSWIe Pp 19 VIIeSsod9U SIIOUIOU BI SIINPal op a[qissod 
4Sd [J ‘(uotstogid op sntd Inod 98-¢g ‘966 BSNOYPooM 
yo yseqsey 3° QI ‘d ‘S66] UleISpjoyH JIOA) aIIOWI9UI op 
don dnooneaq spuewiop JoLyjIyo FJ “a[Qpour fo} un Jaysnfe .p 
J[qIssoduit 3s9 [I ‘aTjanjoe aindY | W ‘JH NeoAlu oWIISION 
NY JUILIVA SJUSIIJJo09 so] sno} JUOP sea} satquizea 
SLT 19919 DUOP SUOADP SNONY “SINIMIIAIOJUL G/Z 19 SJURpuod 
-31 9ZO € SUOAR SNON ‘aWIOUS onbiyeULIOJUI JUDUO}Te Op 
g110eded OUN SPULLUAP JaSTOIO UOTJLIIJISSLIO B IJQPOU! JT 
‘ajueyoduit snjd ev] Iso asAyeue oigiueid 
eR e@ woddel sed aoualajJIp oNeD ‘Uepuodal J] JUASLIQ}OeIVO 
Inb sa[qeiea So] 39 INOMIIAIONUL | & II] DOULLIVA kT 9NUS 
uonoeiajul ounone e A.u [I,nb uaiq Is ‘nesalu OWgIxnep 
9] SUBP BSNIOUT Iso (2) INIMIIAIOUI | @ oidoid sted ey] 


44s "Tassaud d+ 'Tassaud d + 


0 


ELIAWI008 °d + ‘axas |g + v= "u 


‘OJURAINS Z NVIAIU Op UOTFeENb9Y J 
Isule SUOUd}GO SNON ‘a[Qpoul 99 sUep NeIAIU OUWIQUI 
ne JUSNIIS 9S INIMIIAIOIUL,| 19 JURPUOdaI J] ‘JUSUOT[ON) 
-da0u0d ‘Seu SNVSAIU SUWIQISION] UN IUIJEp SUOADP snou 
‘UT VuUIWIeIZOId 3] SUP I9SIOIO UOTILOIJISSeID & B[Qpou 
9g Jajsnfe nog “«[o9I» NVIAIU OWISION UNNe 94SIX9,U [] 
(6) 19 (g) suonenbs sap “/ a '/ saorpur xne juapuodsa09 
SOIPUI Sag ‘asAyeue oJQtWeId ey Inod gsI[WN Injeo onb 
9[Qpoul surgu np sed 113e,s ou [I,nb JausI[Nos op yueyodull 
JSO [I SIVJ[ “INBMOTAIOVUI,| Inod / sdIpul.| 39 JUepUodgI 
9] Inod 2 dd1pUul,| JasI[yN,p suonuyuos snou ‘anno uq 
‘(Ep UONdes ‘(g) UONeNbY) dgsIOIO UONOIJISSE[O & B[QPOU 
np [| neoatu ne puodsaio0s uonenbs a00 “uepuedag 
‘QOURSSIOID OP 9GINOD oUN Issne ajuosaidel [ NeOATU Op 
2[2poul 9] onbsind ‘uonej}ou sUTWIOD w SUOSITOHN snou ‘1d] 


i fy i) fy 
9+ 9GHNNV &+ U= "4 


{QUIQUI BP] 9}SOI | NVIATU op UOTIeNbg,'T ‘~asueyo sfeapoul 
3] uepuedas ‘(ips JUSWIOUL Ne /INSMIIAIOVUL | Jed 9MIIA 
-13}UI 7 yuUepuodgi 9] yUOUUOp onb «sed jes BN» sasuodg1 
Op sIquOU J] BINBWap ajuepuadep sfqeleA eT ‘(¢€ UOT}OAS 
JIOA) o1yNe J B BJOAO UN,p (asuLYD ‘seo sap wednyid e| suep 
‘Jo) Jasueyo ined InoMatAJO\UI | onbsind ‘ayeqoys enbryo 
-IeINY BINJONI}S OuNINL 9jSIX9,U [I ‘aIJUOD Jeg “SINOMIIA 
-IDJUI ST SUBP SJOGUID SIONS 1UOS 19-xNad 39 sjUBpUuodel 
SO] SUBP SdgIJOqUId s1O0DUa yUOS saInsow say ‘ondwiol 
1so onbryssess1y oinjonys ey ‘askyeue sWaIxnep 91199 sued 
‘sIoJ g[nos oun .nb samortAjojul 939 JUO,U Inb sjuepuodar 


612 


Sg] 39 Joued np saIquIOU saIjNe So] ‘INOMOTAIOJUL OUIQUI 
9] Jed stoj xnop samdIAsOjUI 939 JUO INb Joued np soiquioul 
So] :sjuepuodal sap a[quiasud.| Ins osyOd oscTeuR 9139-0, 


SINVGNOdda 
Sad ATHINASNG ‘ASATVNV ANGIXNAd *L 


“SINIMOTAIOIUI o.Ua Nb syuepuodal onus 
ouOJ snd so uOTeUeA eB] ONb aUOU! DOURLIVA LB] Op 9ANLIAI 
Inapurls ve] ‘stOJaynoOy, “JURUOCWIT INIMOIAIOIUI.P JoJyJo 
un 9}stxo [I,nb yuaanolid sayapoul sa] snoy, “ayNe | e a[dAO 
un p «sed IVs ON» Sasuodal ap sIQUIOU NP UOTIN[OAS ouUN ,_p 
aAnoid ouNone UdUUOP ou sTeUIJ sofepoul sou ‘ano Uq 
‘snid wou JeoIJIUZIS JoJJo UNONe v,U ‘Pp J[QpoOUl 9] Sup NO 
2 s[Qpowl o] suUeP AANNV 2QeUeA LF] op uoNONpoyut TJ 

‘sjuapaogid sat onb aisnfe xnarw yso ‘axeyduroo 
snjd jso ¢ NKSAIU Nv DOULIRA LT Op dINJONIs e] JUOP ‘e[gpour 
aD ‘sinNsut sen sjuepuodel say inod onb syinysur nod 
sjuepuodai soy Inod ajueyodunt snjd dnooneag 1sd INSMOIA 
-I9]Ul | @ dT] DOULLIVA FB] :p B[Qpow np uoNeygids9jur one 
oun auesgidal 1990 ‘a][NU Iso INaTeA 9}}99 IS aJUROdUIT 
snjd elas 39 QILIeJOOS ap NVOAIN SANeOTTdxs s[qeUeA vy 
Op ANg[eA P| 9p puddep INIMOIAIOIUL | OP NSAIU Ne DOULLIVA 
RI ‘a[Qpoul 99 sue] ‘dD BJEPOU Ne JUSWISATeIEdUWIOD JUSU|Q 
-BIOPISUOD SJUSWISNE INIMOIAIOIUI | op NedATU Ne MENG 
Op ANg[eA LI ‘p sJapoul sj sueq “jueyOdu snjd vsas syINysUT 
snjd soy 19 S}INSUI SUIOW sa] sJUePUOdI so] aNUS URd9 | 
‘«sed yes ON» Sasuodal ap siquiou pues snid un yuaT[IanseI 
Inb sInoMorIAJojUI say Inod ‘juonbgsuod seg ‘opus 
snjd onjosqe InojeA oUN OUOP LIN 19 SINOMOIIAIOJUI SIO 
Inod snd a109u9 eIVS Of [I ‘JHeSOU 3SO QILIR[ONS Op NesAIU 
a] Inod axl eneuresed a] onbsing ‘aylejoos op neoalu 
9g] Inod a] qrey snyd JUaIdIJJo09 UN JUO d9gAdI[9 1S9 9]ULISUOD 
RI ap AngyeA ey] sjanbsay Inod sinaMoIAJOJUI Soy Wp JUOWIONN 
‘QATeBOU ISO DOURLIVAOD BT “(66(0'P- = eo) 
ajueyodull jso giLIe[OOS op neaatu a] Inod npIsel 9] 39 
Q]UR}SUOD PT INOd NpIsel o] a1JUS VDULLILAOD FP] STRUI ‘SATIS 
-1j1u3Is sed 4sa,u we Io ap INI[VA BY ‘NeAIU DUIVISION 
NV I[QULIVA JS9 JUDIOIJJ909 9] JUOP aINas eI Iso sJURpUddepuT 
JIQeLIeA 31105 ‘sjuepuodel sap ayLIe[OOS op neoAlU ne 
g1o0sse ajQuresed np UONeLUeA oUTeLIOO oUN oUOU asATeUR 
ANON *(Q BJQPOU!) SINAMOTAIOJUI SAP NRIAIU Ne sarlojege 
sojuod sop Jajsnf{e.p gXessa yuowaess suoae snoN 

‘sjuepuodal sap NedAIU Ne DOURLIVA P| op aJURWOdUIT 
UOTNUIWIP uN Issne jInpoid syuepuodal say JURSLIQ}OVIeD 
SgjqeueA sop UOISN[OUI,T “(9 s[epow) syNeoIyIUsts 
XNap SNo} jUOS Zassaid ap ya [assaid ap sjajjo soy “«sed jes 
ON» sIpuodal ap saouey) sey yNpel assaid ey] suep sanbrtjod 
SO[[PANOU SI] SIAING “a][QIeJ ISO QILILTONS ap NeIAIU 9] JOP 
xneo onb juswmenbely sulow ‘sjInsUT Seq) syUBpUOodsI so] 
32 souWOY saj onb yuourwanbery snjd «sed yes aN» UoNdo J 
JUDSSISIOYD SOUT] Say ‘asATeUR | UOTSS “(100‘O>d) sanevoy 
-IUBIS JUSWID}0U 39 DJURLIOdUII 3So J BOT Z- op ANIIRA P| 
Op UOTNUIWIP eT “o[gpour np yuowa}snle | JUDWIATGQeIOpIsUuOD 
JUSIOI[9UIe JURpUOds Np sanbNsUg}deIeD So[qeUeA Sa] 


000Z eiqwecegp ‘ajanbua,p senbiuyoe | 


9 
89°C6Er 
(696'0) OSZ‘EI 


(ZEE‘T) 769'8 
(618'T) 96€'T 


(918‘L) 660°r- 
(601°Z) 060°9 


v 
TS‘ PI by 
(6960) OLZ‘EI 


(80€‘L) 6£6'8 


(6960) 999°7 


I 
7981S 
(pL6‘0) OOE‘ET 


(ILL) O8rrI 


(€v0'T) 1S7'7Z 


 9[QPOU! Ne JUSUIOATeIedUIOD , :BION 


PV 
SEOISp TIC 
(vL6°0) OZE'ET °0 


(VILL) OLP‘rI 


(OvV0'T) 6r7'7 


6 


SOINSOW Sop NVSATN 


aqueIsu0s, 
xe) 


syuepuodgi sop nesaIN 


PITPIOIS 


G 
Que 1suod/pILTe[OOS 9 


ayueysu0d 
rg 


9 


SINIMOIAIOIUL SOP NEAAIN 


s110}e9TV 
(vETZ‘0) 9ZH'T (9€7'0) E8h'I Zassoid 
(€€7'0) L88°0 (€97'0) 1160 [essaid 
(9tb‘0) 8LL‘I- (Stv‘0) SL9'T- 91Ue[00s op neoAtu 
(vIv‘0) 8sr‘z (vEv‘O) €6€° Z 9axos 
sjuepuodgs sap neodAIN 
(680‘0) ZL0‘0 aguue 
(€7S‘0) PSL‘E (Zrr‘0) 6bL‘ € (8S€‘O) 820° (ZZE‘O) 9ET PD 9)UB}SUOD 
SOINSOUI SOP NRIAIN 
axl 
P 212poW 2 2]@poW q a1apow @ a1@poyl 
(sasayjuored anua *j-"9) INAMITAIOIUL OUIQUI 9] Jed SIO] XNAP SPMdIAIOIUI 919 UO INb sjuepuodg sap askTeuy 
z neaqe 
SOJUBAINS € NBVSATU Op uonenb9 | sure SUOUDIGO SNON 
“sduio} 


OP I[GeLIVA SUS O[QPOU UN ddAe SIAINSINOd SuUOp suOCANOd 
SNON ‘«Ssed jes dN» Sasuodal op siquioU Np JHeOTJIUsIS 
[eqo]3 jusuIesueyo uNnone ne e A.U [I,nb ainjouos suoanod 
SNON ‘(q B[Qpowl) snjd uoU JHeoUsIs sed yso,uU s[qeLeA 
RI op onowresed 3] 39 oANeOIIUsIS sed jso,u (J 307 Z-) 
uUsXOUL PF] & Ued9,] op UONNUTUNIp wT ‘sisnfe xnorwW sfapoul 
un sed yinpoid ou sguuy s[qeueA ev] op UOISN[OUI,T 
“sonne .p aonb «sed 
11S OJ» sasuodgi ap snd nuajqo jUO SINOMIIAIOJUL SUTLLISO 
39 sone .p onb snjd «sed yes ax» asuodal ev] gsi[N JUO 
sjuepuodai suteyioo ‘anne | & a[oA9 uN, p ayInpoud 3sa,s uoly 
-PLIVA OUN ‘YIP JUDWIANY “SOATBIIJIUSIS JUOS SOOULLIVA SI] 
SNOT, “INIMAITAIOIUL,| B SOT] aJUeSOduI0D ouN 39 JURpUodsI 
ne sgl] aJuesoduioo oun ‘sainsoul xne sgl] aJuesodwo0s oun 
pusiduos ayuepuadap a[qeLiea eb] op VULUA PJ ‘SIOJoINO], 
*JURISUOD 389 «Sed yes ON» SasuUOdal op sIQUIOU 9 ‘a[Qgpout 
99 Op sed a] sue ‘sJUepuodsI sop IN[ed B IU SaINSOUl 
SOP NRAIU Ne IU sajUepUuddgpuUT saTqeLIeA Sop JUSTJUOD OU 
I[QPOUL JD ‘[Nu sfapoul ZI] 1Ss9 Z NeITqGe} Np & s[gpoul aT 
‘JUSAINS INb JeI[NSgI Sat SUOUD}qGO snou ‘uTjy oUTWeIZOId 
3[ suep J[Qpow Np suOoTeoIfIogds sod suOUS sNoU Ig 


f { 
<0 6l0y, , Wg _ fg 45 py 4 Oy = Oy 


‘saltoyegye sajuad sap 39 allojyege SUISLIO.| 8 B9UUOpPI00N 
aun suojsnf{e snou stew ‘asreyuourgtddns ajqeuea ounone 
SuON OUI .U sade ‘(SINOMIIAIOUL) NVIAIU SUIZISION NV 
*“'u Inod orreptuuts uoenbs oun suoine snou 
‘JHeOIJIUSIS so SQUUR | B QdOsse JHeUISS oQUIeIEd ZI IS 


01. cassaad Yd + 'Tassaua *°d + 


‘auravioos “9 + ‘axas Og + Ou = “mw 


:sayqeuea anenb yuanU0s 
Z nesalu op uonenbs | ‘suog ‘assed el ap UoNeINsuOd op 
SO[GULILA XNAP Sd] 39 9JLILTOOS OP NLIAIU 9] ‘9XOS J] IIOAS B 
‘SOTQLIVA SION SUONTOUI snou ‘s}uBpuodal sep NesalU NY 
“INIMIIAIOIUL | B 39 JUepUodsI 
ne soidoid yuos [ neaalu ap uonenbs,| ap syUatoIjJoo0o 
SY] ‘d]ULISUOD INITeA SUIQU OUN Pp Se}odaI $}S9} Sop OUIUUOD 
SOINSOU XNap Sa] JaIgpIsuod sJoye suOANOd snou jo sdus) np 
sinoo ne sed area ou «sed yes ON» sasuodgl op SIQUIOU 9] 
Janbey uoyas (Tp ‘9661 SIOplius OA) «jfIeU s[Qpou no [NU 
J[QPOUl UN yUATAap s[Qpoul ZI ‘sed 3s9,[ OU [I,S ‘JHeIJIUsSIS 
aug) un ysa “!y Is youTULIaJep Inod jsaj un suonbydde snoyy 
(S661) € NO (Z66I) O INaTeA eT eIpusid ynod ‘MOTAIOJUT | 
nol] no & No sguurR | 3Sa Inb ‘(s) sdusa} op sfquueA SON 


jeued sed saygnbua sap seo a] SUEP INAMAIAISJUI,P SOHO SOP UOI}ESI|BPOW\ -]P|SASOO7 Je Kiayold 8Lz 


a Ay fio fy 
2+ FHANNV pee fit eal oy 


:OJUBAINS 
I nesAlU op UoTeNbg | Isure suOUd}qO SNON “[ & INOLI 
-gdns ong ynoed ou swigudjod np 918ep 9] ‘sounsou xnop Ins 
onb ayod ou asAyTeur a0U SUTWIOD (f A) f ANIMATAIORUL | 
Jed 9MdIAJOUI ‘47 JUOWIOU Ne 1 JURpUodgI 9] Inod «sed jes 
ON» Sesuodgi op s1qUIOU 9] IS9 aJUepUddep s[qeLeA ONON 
"S9[099 SI] SUBP SdAQIP SOP 
OIIe[OOS JUSUOPUSI NP UOTNIOAY | JUasATeUR SINayNe soy ‘oT 
-Wdx9 399 sue (7661) YSNquapney jo YAIg Op s1IeN Np g 
anideyo np sjduiexa | & [qu quias Jso a[QpOU A_ “aINSIquie 
sues so onbiyosesoIYy SINIONS VT “SINOMOIAIOJUI SOT SURP 
S9IJOQUID SOUIQUI-xne ‘sJUBpUOdaI Sa] SUP Sdg}TOQUIS soins 
-OUI :XNVOAIU SION & «a[dWIs» a[apowW UN .p UdAOUL Ne jIeJ 
Qs askTeUR 9939D “(] NeIIqQeI NP | ILI039}e9 e] SIIP-e-3S9,9) 
INIMIIAIOJUL SUIQUI do] Jed SIO} xNIP SAMOIAIOJUI 939 UO 
nb sjuepuodai so] Ins onb oyod au asAyeue algtuold eT] 


TAVATAVL Nd SLNVGNOdAY Ad 
T aMWOOULVO ‘ASATVNV ANGINA ad = °9 


“INIMIIAIOIUL | Op sonbysigyoese9 
SOP UOTOUOJ UD JOJO 199 Jonbi[dxa .p JoAessa sues “INOMIIA 
-I9}UI,p JOJJO UN 9}STxO [I,nb yuawazduts suosoddns sno 
“INOMIIAIOUL,| Op sonbysiig}oesed sayqeueA op sed suosinp 
-OUI,U snou ‘asAyeuR | Jonbijduros don ap JoA9 INOg 
‘aygnbus .p afoA0 np neaalu ne dInsoul 
R[ Op JUOWOUW ne USUIOLIOJ do a9g[9LIOS jeIIS s[qQeleA 
91109 Jed ‘a[epoul os] suep URepUOodaI np o3e,] sed suosInp 
-ONUI,U SNON “dINsoW s1QTWAId e] Op sed a] SueP ‘apUodaS 
RP] 19 DINSOW SUIQIXNEp LI Op sed 9] SUeP sJURpUOdgI soy 
$no} Inod anu so a1gtuaId eT] ‘oWIgIxnep 9] nod ‘zassaid 
39 9[9A9 JomUeid a] Inod [assaid MOAesS ke ‘soTqeieA xnop 
Isure SUOUS}GO SNON “(ZZ “d ‘966 Ssopfius 110A) ouUUdAOUT 
INI] 3S9 safqeueA xnap sad Inod soUsIZJ9I Op INo[eA 
eT] ‘agsod 939 sed eu uonsonb ze] Is oinsoul ey op JUSUIOU 
ne Q Inaea e] Jonquye inj ap sind ‘ainsoul op jyusuIOU 
onbeyo inod ({ soueuea ‘Q ouUcAOU!) JasI{eULIOU eT e 
9ISISUOD B[QULILA OP BIUIS 99 JO}IeN) Op USAOUI J] “a[Qpow 
Np JUoUWassi[qei9,| odeJURAEP dJODUS onbi;duI0D suoNsonb 
SOP QOUOUY,| OP ILIL[IWISSIP 0D “oxouUe Ud JUWITeSO 
jusINsIy assoid ev] Ins uoNsonb eI op sajueLeA xnop so] 
“Q[DAO OUQIXNap Z| Inod sIjIpow JUDWUAIES9T| 939 e UOTsONb 
RP] ap gououg,| isd snd inb ‘sdwia) np uoNouoj us ateA 
Inb s[qeUeAOd DUN sd SUIZISION eT] “oINSOUI op s}]USUIOUI 
XNop So] Inod sajuLJsUOD JUOS Sa[qeUeA Solgtueid Xnep sa] 
‘(sreuref = ¢ - sinofno} (anbsaid) = | :essaid) assoid ey suep 
sonbnijod say[aanou s2J JaAINs sjuRpUOodal soy aT[onbe] suep 
SINSOUI LT 19 (9AITP = | IIQIeJ = CQ) 9ILIe[OOs op Neaatu J] 
‘(QUIS = | ‘OUWUOY = ()) Oxas J] :soyURepuddapuT sa;quLeA 
SIOI] SUOI9PISUOD snou ‘sjuUepuodgl sop neodAlU NY 


Lle 


“oWgTxnep 
3] Inod ston e [e389 39 aA JatwaId a] Inod o19z e [e39 
tos sduray a] anb 9110s ap s[qQeUeA 37399 9POdEI SUOAR SNON] 
‘QINSOUL Bl OP JUSWIOW NP INIJVOIPUI SUTWIOD MOIAIOJUL | 
op s9UUe | SUOSI[N snou ‘soINsoUl sop neoAlU NY 
“sjuepuodal sap s[quiasus | Inod garlosqo usAou! s1quIOU 
3] onb gazqje snd 3so syuepuodas sao Inod s[9A9 owerxnep 
NP SIO] austZoluse «sed yes dN» sasuodel op siquIOU 2] 
tonbinod sanbijdxe ,p yauied ou uary “WUsWATIEdSal ‘7p 
OP 19 6‘€ OP ISA USAOW SIQUIOU J] “INSMITAIOJUL OUIQUI 9] Jed 
SIO} XNOP S9MIIAIOIU! 939 JUO INb syuepuodal soy Inog "p‘¢ & 
WeIJJIYO os Joued np saiquiowl syuepuodal sa] Inod suusAour 
RI 12 OE ap 1e}9 «sed jes BN» sasuodgl ap udAoW [eqol3 
aiquiou 3 ‘a[dAo swWgIxnap ny ‘suoNsenb sap si[NdIJJIp 
R] op (no)}o aJgnbua,| op yafns a] nod yg19}uUI.p onbuew 
np osned e& ‘a[dA0 WgIXNep Np sIO] sgjUasgidol-snos 
juatos «sed yes oN» asuodal ve] op «sordnyjnu sinayestpyNn» 
soy onb 90 & oipuone,s weanod uo] onbsind ‘sjueu 
-oidins sed yuos ou sje][NSEl SdD *(Q‘E) INSLIDJUI JUSUIIIES9] 
yso uofou siquiou 2] ‘Joued np saiquiow syuepuodel 
$9] JUDWIONbIUN SUOISPISUOD SNOU IS *g] INS SIOJ JnoU SUTOW 
ne uojdo 3199 Istoyd yuo sjuepuodel op % QZ enbsolg 
‘yuepuodg1 sed anenb e inatedns yej9 «sed yes oN» 
sesuodgi op usXowl aiquiou 9] ‘a,oAO Jomweid np sio’y 
‘(1861 Jessoid 
ya ueunyos siduioxe Jed ‘I10A) sIquioU 990 JajUdWIsNe 
assey oM9I[dxo any a] onb 90 z Issne suOIpUaye snou sno] 
‘QYO¥) LP] Op 9Uxa]dui09 e] op asneo ke ‘«sed jes aN» sosuodal 
@P eIQeJapIsuod sIQUIOU UN II][Iende1 B SUOIpUs}e snou 
SNON ‘oxouUe Ud dINSIJ UOTsoNb ve] op yo[dwiod gouoUs TJ 
‘juepuodal ne sgUUOp XIOYS SIOAIP So] JURALIO9P oVed 
R] Ins QuUONUSW sed 41e}9,U I9-IN[Id sreur ‘(sed yes ON) 
ayordxe any un yeyodwios uonsanb ey “aurstjesapej 2] 30 
onbmwo0u0dg sUIsITeIOI] 9] ‘OUISTOITOYIS 9] “UaWOATOOdSal 
‘yonjeas Inod ‘yuepuodgi ne sagjuasaid 939 yUO sal[eyD9 
SIOI], ‘sjUIOd [| & SaT[aYd9 SASIOAIpP op UdAOUL ne sanbrtjod 
sted XIs Jon[eag & JUATeISISUOD SaTIOTJJIp sntd soy soyoRd 
Sop oun | ‘a[e10}09[9-}s0d aygnbus | xp MoTAIOIUL | JUeING 


SHUSATVNV SAUTHVINVA °S 


‘oskyeur .| SUBP SUOSTTIN snou 
onb sajqeuea soy [lejap ue snjd suoumuexe snou ‘aqeieoid 
NY ‘NIMTW/UTW XneoAlUN[NUL UOTesI[gpOUl ep [aI918O] 
np ueXow ne sanbiydde ang sno} juaaned inb ‘soyepour 
SIO]] S80 OP SUOAIOS sNOU snouU ‘/ 12 g sUOTIDaS xnY 

*xne39 91}9 JUDAIOP SJULI[NSaI SOOURLIVA 
ap sajquiesus sot onb uoNIpuod ey] Josod e ya Inowedns 
NRIAIU NV BLOJVY[e JUSIDIJJO09 uN apassod sajOIJ B[QuLIVA 
onbeys onb Jasiogid e ‘UONROIJIsse]O one] Op gun 
onbeyp inod anor ajqeiea oun Itulyap e sind ‘adAy anbrys 
-IVIQIY UOBIJISSe[D DUN & JUDWIDWIOJUOD SUOTILdIJISSe[D 
Sop dUN,| Bp suONRdTJIONds sa] II[qeI9 B dISISUODD spoyeU 
ey ‘ody xneoalun[Nu [storso, uN.p (juaWUeNbgsu0d) 
yo onbiyoreisty yuoweind uonejnuoy oun,p udaXoul 


000zZ aiqwasvap ‘ajanbue,p senbiuyoe | 


NY SaQpoul sao JoWINse 39 JolIOeds JUSWUIOD JUSNUOUT 
(pZI-EZ1 “d ‘S661) UleISpIOD 32 (P66) UlaIspjoy 19 
yseqsey “JowNse sof inod pry swiy ose | sp uonesyyN,| 
12 So[Qpour op o1Uas 90 ourWIeXa (¢66]) YSNquepney 
“(in yo ““n anuos “nyo ‘n) sywepuadapur quowrjanynut 
JUOS (INOMOTAIOIUT,| & Jo JUepUodg1 Ne sol] sNpIsel soy 
101) SOUDIOJJIP UOLOLISse[O op SaimjonNs sop inod snus}qo 
snpIsgi seq anb axjno ua asod uo ‘ajgpour 99 nog ‘uONeS 
-IJIsse[o op omnjonns onbeyo nod npisez un yuo x ajuep 
-uodopul a] qeLea ey & sgtoosse somguiesed say “*/ noMmorA 
-Joqut | Jed gmotarajut ‘'/ yuepuodgs 9] Ans oye 7 oInsoUT 
Bl Op Angfea eT ‘tor ‘omtp-e-ysa,9 “L149 '/ uojas agutsseyo 
‘1 gulun,{ inod agasesqo snajea ey aquasgider /'47 
‘(¢) uonenbs | anb 
uo0dej OUIQUI eI] Op sg[NULIOJa1 ag ined (6) uoNenbs,T 


(6) “Ay i. ty x rng 3 Ag 13 On f On K og i tg 


(9) TMi, , Why Tig , Mog _ 0H, 


{(SOJOUTSIP UOBIIJISse[O op somnjonsjs 
xnap say queusisap “/ 39 'f saorpur sap) wns Inb su0] 
RB] PUdId SgSIOIO UOILOIJISSEIO & S[QPOW UP) “INIMIIAIOUL | 
uojes 39 jJURpUodsI a] UOTIS sogsse[a yUOS soINsoU 
so] ‘o[dwioxs aj0U sued “JUSAIA S]I No Jansenb a] uoTes 
no yuojuanbaly sji,nb a[009,] uoTas sasse[o ag yUdANod 
SoAg|a Sap ‘a[dwioxe eq “(OLI-EII “d ‘S661 UleIsplon 
JOA) g[QeLeA ouN.p snjd ap uONOUOJ Ua sagssE[D ang 
judAnod sgylun souleyag ‘anbryosesaty yuouroind oinjonsys 
oun 9)n0} sed jU0,U sagUuUOp Sa] “dgSIOID UOTIBdIJISse[O 
@ B[QPOUL 9] 3S9 sUOSITNN Snou anb sjapow sWISION FT] 


I9SIOID UOILIUISSE]D & SIIIPOY. ¢"f 


"So[QeLIRA NO (SINSOUW Op S}USUIOU 
$2] Sno} Inod saurgUI say) saJURIsUOD ag JUSANOd seT[onp 
-IAIPUI Sonbyjstig}oesIed say “(Z NedAIU) 9AdT9 SN{d NesalU 
UN .p JUOS X SaTqLLIeA Say ‘a[epoul ed suep ‘onb J9}0u op 
STOJOINO} JUSTAUOD I] ‘(UepUOodeL) saT[aNnprIAIpUI sonbysug) 
-deIeD Sa] Jayuasgidal Inod x suOssIstoyo snou “(Z) & 19 (T) B 
JUSUIPULIOJUOZD “syuepuodal sap s[quIosud,| INs JasI[eIgUeS 
ep ouop ‘anbrytoads atanpraiput ared ey aiinpel op afqissod 
4SO [I “(X) SoyJonprAIpul sonbysugjovsied op UOISNOUI,| & 
a0RIn *(°7) Janprarput nprsai un + (x) perguas anguiered 
un e sjuajeAInbg jUOS sjanprlAIpul sonourered soy] 


(L) A074 aoe 4 Oy — Oy 
no 
(9) 10 pee y= Oy 
:sanguivsed sao WUIJep sfepow np otued 
sUIgIXNap wT ‘ane, | & gIUN sUN Pp JUIEIJIP sanguiesed sao 


anb astogid ajapoul a] ‘(uepuodgs) gy1un | ajuasgidaz nb 
SOIPUI UN VoURSSIOIO op soNQUIeIed xne Issne sUUOP UD 


jaued sed sejanbua sap seo a] SUP INSMAINSJUL_P SJEYo SEP UO!ESI|BPOW\ :}P|9ASOO7] 18 Miaxdld 


'¥Y 9139p 9p s[Qpowl 9] anb sey Ns9I sInoT]Iow! yUoUIS}}OU 
op ouUOp | + ¥ 9139p 9p I[epoul ZI Is JIOA Inod essa] 
o1lej ynod UGC ‘9qINOd eI Op ojURSIeJsHes UONEIUASEIdoI 
ouN QUUOP 19-IN[99 Is gIsap s[qrej ap owouAjod un Jost] 
“190 .p jURSsaIg}UI snyd jso [I “UOWIWOpIAg ‘aqinoo ey yUOUT 
~9108X9 BIajUasgidal [| - w 913ap ap ouloUdTod un ‘amnsow 
OP SJUSUIOU UW 9ISIXO [IS “ITDUT] ISO 9QINOD BJ 19 | B [eB 
4so y ‘o[duts seo un sueq ‘awioudjod np giZap 9] sjuasgider 
¥ 30 ‘1 yolns a] Inod souessto1o ap samouresed no 9aII0} 
-sofery op songuieied sof juos (¥°"Q = y) “a say ‘a3e,] no 
SINSOUI LI VITVJ ISO NO JUSWIOW 9s] ag JURANOd 7 ‘47 JUSUIOU 
ne 7 yWepuodsg aj Inod agalasqo InayeA ev] aJUasgides "7 
(S) He a print PCO or eu a. yu se 0a 3: + 

:OJULAINS eT ISO UOTJeENbg | Op spetWIOUATOd sULIOJ 
aun & 9qinod e] enb asoddns uo ‘4uoweyjemiqey ‘oinsour 
91199 Op sol[enjUueAg sUONeULIOJsueN xne nb tsure ‘a3e,] 
d0AR NO sdul9} op dINsaW oUN dade JoproUrOS ynad Janbey 
‘QINSOU BI OP JUSWIOW Ne sgUUOPIONs sd VdAJOSqO INI]eA 
onbeysd ‘soy[enprarpul sonbystigjoesres say uojas oqinoo 
Rl Op UONLUPA PI ISIPNII,P 39 SOINSOUI SASIOAIP Sa] Side .p 
SOURSSIOIO OP BQINOD dUN JOUINSS,p nq Inod e asATeuR TJ 
‘syuepuodgl sap yuos sayun saj ‘adnooo snou nb seo 3 
sueq ‘sked sap no sajoog sap sUTOD ‘sg}IUN SsaINe Pp IIe ,S 
JUSWIST[OINJeU WeLINOd [I steur ‘souUOsIad sap jUOS SOT[ONPIA 
-IpuI saytun say sduia) np wednyd ey] ‘ourerxnap 9] ‘sogmnsoul 
SOUUN Soy 39 NVIAIU JoIWoId 9] JUD}USSgIdaI saInsoU soy onb 
alepisuod uo ‘anbryosesJety sJepowl uN ,_p ueXow ne «sagyede1 
SOINSOUm Op aSATeUL | OP SIO'T ‘[PUIPNISUC] xNeoATUTTNUI 
Q[Qpoul a] ISO suUOSI[IN snou aonb sfepoul swigixnep a] 


a VUIPNjzIsuo] 
asA[eue | nod XnvaAluy[NUl 3Ipoy] Zp 


"Sg[PLIOO JOS SII ‘JUaWATTaNIIQeH “one | op uN | syuUepued 
-9pul ang sed JUSAIOp OU SNPISAI SdD “7 BOURLIVAOD Op 
DOLIVU SOAR SUNUIUIOD SPUBAN[NU O[PULIOU IO] oUN JUSAINS 
sjtnb 39 uP op sjuepusdapul ywos ( "199 %n) Z NesAlu Op 
snptsei sat onb ano ua suosoddns snoy ‘sjuepusdgpul juos 
( M5) ] NeaArIuU ap snpIsal say onb suosoddns snou ‘yJosoINs 
aq ‘x 12 Z SoATeOT[dxe so[qeueA sop sIno[eA so, guUOp 
yue}9 ‘a]J[nU suUeOW oUN yUO “a ja My «Op SNPIsel So] 
onb suosoddns snon ‘suonenbs soo suep Z 39 X SolqeueA op 
siquioU puess snjd un Nour WeLINod Uo ‘USWATTAINIeN 


hi i f fi i 
(p) My < On ep My Ny 4 Ay My tty , Mz 10, , Mylg Zi og = fy 


:JUBAINS 
[eqo|3 s]Qpoul s_ suous}qo snou ‘([) suep UONMsqns 
zed (¢) JUesINpoNUI Ug “2 NVSATU SUTVIXNEp op s[qeueA ouN 
SUOAR SNON ‘SINOMOIAIOIUL SOp s[qUIosud,| INS sUONRSITeI 
-9U93 So] aIauied nod (InematAIO}UL,| ap SonbySLUIg\OeIED) 
nvaatu yney snid op sa[quiea op g sjuatd1JJ209 sa] sIpusdap 
airey tssne jnad uQ *(''m no n) inaMmatAJOJUT J B Ot] NPISZI 
un 9)SIx9 [I ‘g ap Ingfea onbeyo nog ‘anne | & INBMITAIOIUL 


9le 


UN .P JALIVA g SJUDIOIJJ209 sa] “INOMOTAIOUI | & soidoid 
JUOS sajgueIed soy SIVUI ‘AITBUIPIO UOISSSIS91 Op o[QPOul 
UN & I[QUIdSSaI UOTLIA1 9139D “JUepUodal 99 op x o[qULIRA P| 
op puodap ‘/inamotAsoqui | sed 9MotAJojUl ‘2 JUepuodal Np 
A asuodas op d[qeuea ve] ‘OUOd “INOMOIAIOIUI | B ‘Z NESATU 
3] 19 JUepuodgl ne puodsa0d | NedAIU JI ‘OJ °Z NVDAIU Op 
guiun J ‘/eo1pur.] 9 | NBSATU op gyTUN | OUBISEp 7 SOIpULT 


(Gye rth ee al pen Ig = Ng 0m 4 Mz OK 4 Og - fog 
no 


(Z) Md age "Gao ae (deg 


(1) Be yd ed oney 


:JSO QULIOJ B] JUOP ‘[eJ9UES XNeoATUTTNUI 
Q[Qpour a] 3S@ suOSI[N snou onb sfepowl Jonuald a7] 


[¥19U9S XNVIAIUN[NU IIpo| =p 


SASATVNV SAT XNnOd 
SASITILA XNVAAINILTOW SaTAGOW 
SINGTYAAAIG SAG NOILdIMOSAd AATUA “PF 


‘yins Inb uoNsdes 

RB] & sudjUasaid snou anb sjouNsIp sayapour SiON SUOUTQUIOD 
snou ‘axa[dwi09 ainjon.s ap saguuop sad JosATeue INog 

‘(YW 911039}89 P| Op xnoo eB JUSWIaITeUOD) 

slo} xnop juepuodel swiguI un 9MOIAJOWUI siewel 

UO ,.U seul ‘sayoAo xNap xne gIOQRI[OO 1UO g 911039}e9 P| 

OP SINIMIIAIOJU! SAT “(C{ BI0B9Ie9) afOAO ouIaIXNnep ne nb 

no (Dd e03891e9) afoA9 Jared ne nb gioqey{[oo yuo ,u Inb 

Xnoo 19 (q 10 YW $9l1039}e9) aJgnbua | & SIO} xNop sJOQeRI[OD 

UO INb XNdd OARS & ‘SINBMITAIOJUL SBT aWUS UONOUTSIP 
RI alle} suoAnod snou enb Issne ayuOU nes{qe) a] 


(syuepuod;y) N 


SINIMOIIAIOIUT ,P 
d a10391eD 
SINAMOIAIOIUT ,P 
D eH0891%D 
SINIMIIAIOIUT ,P 
g e110391%D 
SINOMIIAIOIUT Pp 
VY a089}e5 
aoka 


LY 


(SINIMOIA 
-J21U]) N 


syuepuodgs 
ap ap ap ap 
pawosg7eg ¢aosNeD ZaOspeD | a089IeD 


sjuepuodg1 sjuepuodg1 sjuepuodgs 


soguuop op a] quIasua | op UONeJUsgI1d91 ou) 
T neoqey 


‘(nes[qe) Np p & | SaLI039g}v9 sap syuepuodsl) syuepuode 
8ZO € Sep s[quiosud.| Ins ood osATeuR sWIxnep eT 
“INAMOTAIOJUL OUQUI J] Jed SIO} XNOP S9MIIAIOIUI 939 JUO IND 
sjuepuodal p6¢ onb a}du09 ou sIQIWIEp 9119D *[ a110391e9 


Sle 


RI Op [20 & puodsa0d uoNeNis e] JUOp sjuepuodal soy 
suOUTWIeXa SnoU ‘argtuald ev] ap sped 9] sued ‘s}uepuodel 
OP SoIOS9}e9 saUdIaJJIp sod Ins JUSpUoJ as sosATeUR 
xnop SON ‘(p NO ¢ SaLIOZgIed) stoj s[nas ouNn,Nb soMoIA 
-19}UI 939 UO ,U INb syuepuodgi sa 19 (7 BUOS9}ed) sJUSIOTJIpP 
SINOMIIAIOUL XNop Jed sgMoIAloUI 939 JUO Inb stew 
‘ajgnbua | @ stoj xnap edionsed yuo inb syuepuodai soy ‘(| 
D1IOZ9ILD) INOMOIAIOIUL OUIQUI 9] Jed SIO} xNOP S9MOIAIOIUL 
919 Ju0 Inb joued np soiquiow sjuepuodal say :syuepuodel 
ap sednoiz sion oajsixe jI,.nb anuow negqe a] 
“saguUOp op s[quiasua | suep 
B[ONIUSAY JOUSLINIDO uN aJUasgIdal Nes Qu} 9] SUBP dINIIy 
inb sjuepuodal ap ati039je9 anbeys “(997 I + (7 X Z9OL I)) 
SUOTJLAIOSGO (GL p WUEUOD saguUOp ap ye[dWIOd s[quIes 
-Ud, J ‘UONRAIOSGO oUN ajUasgIdeI nes{qe} Np x onbeyD 
‘(pL ‘d ‘S661 UleIspjoyH Issne MOA) [ neafqe} ne s[qei 
-eduioo neajqe} un suep saguUOp ap e[quIasUd | ap sINjons 
R] Jajuasgido1 ynod uQ ‘ainsoul snes ouN.p ‘(99Z 1) 
Q1S01 a] Inod ja ‘sajoAO xnep sap SIO] sayeJ saInsow op 
suosodsip snou ‘sjuepuodal Z79/ | INog ‘(ZI [+€91) SINIMIIA 
~IajUl CLZ B19 (LEE + 169 Z) Siuepuodgl 870 ¢€ B JUepuod 
-SdLI0D SogUUOP op s[qQUIdsUd UN IsUIe suOUd}qO SNON 
“g[OAO SUIVIXNEp nV 
odrored uo siIndMOIAIOUI XNVSANOU Z| [ ‘JUanbgsuod Jeg 
‘aygnbua | e edionsed nesanou ap yuo 9[9A9 Jared ne edio 
-wed juareae Inb sINaMATAIOIUL CC S[Nas ‘jno} U_ ‘(esepuos 
ap uvjd 2] Ins g[Ieyap onbruyos) yoddes un inod ‘1 66] 
‘MNOPasUAMS Jd UOURD ‘JI[I[Ig ‘UoLIIeg IIOA) syuepuodg 
xnevsanou /¢¢ Jo Joued ne yueuoyedde syuepuodal 79/ | 
JUITJUOS UOT[HULYD9 19D “SINOMITAIOIUL LO] Jed S9MOIAIOIUL 
912 WO sjuepuodal 660 Z ‘asATeUe dIQIWIOP UW “1661 
ud sguUOT}OeI9s UNA snd ev] dIOYOO e] Op JUSUASSITIIOIA 
3] Josuadwiod Inod syuepuodg xnesanou ap guUOTe[9S 
e uo sind ‘sjuepuodg: [69 Z op odnois np smyed e 
uO] HURYSE UN JoUUOTNeI—aS Jed 9OUDUIUIOD B UO ‘OUODG “a[DAO 
awetxnap 9] Inod yHed snd uoT[HUeYDSE UN JoUUONDETES NITeJ 
® [I ‘soulejaspnq soyurenUuod say guUOp juLIq ‘o[qeIedwio0s 
g]e10}99[9-1sod aygnbue oun gsijeal & UO ‘G66 BP SUONIITP 
sg] Saldy “(€661 UsUE0g 129 I9IT[Ig ‘MNOpasUAMS ‘UOLIeD) 
SINIMOIIAIOU EQ Jed sgMdIAJo\Ul ‘spueUeyy syuepuodg 
169 Z ep seidne ‘apuewely uoIssl ve] suep sol][lonool 
SogUUOP Sz] SUOSITN snou ‘aoe yuaseid a] sued (S66 I 
dOId/OdSP) sue pL & 8] op uonendod ey ap jneyuasgidas 
ySo UOT|HULYDE J “(PrI-I pI “d ‘7661 UPWNOIAA 19 UOSsSUaMS 
‘Tepuses ‘ajduioxe sed ‘I10A) gigpuodojne sgizep xnep 
e uejd un jso a8epuos op uvld J] °766] Op Slow siomWaId 
Sop sinoo ne sked np suOIs9l SION So] SURP SagsI|eaI S9}DOIIP 
SMOIAIOIUL pps p JuLIdUIOD seuONeU ajgnbua ouN 9gsI]e9I 
® UO ‘anbI3[ag US [66] OP Sa[e1oUas SUOTIDEIP Sa] sdIdy 


SGaNNOd SAdd 
FANLONALS WT Ad NOILdIMOSAd = *¢ 


“sasA]Teue SOU op 
ajuepuodap s[qeuea vy] So «sed jes ANY» Sasuodal ap aIquioU 


000zZ aiqwesdap ‘a}enbua,p senbiuyoe | 


YT] ‘SINOMIIAIOWUI Soy onb Isure syuepuodg: say osAyeur | 
suep dIN[SUI Wop «sed jes ON» sasuodgi soy ronbrydxa ve 
ayorayo Inb sayetoos saouatos sap aystqeroeds un “juanbgsuoo 
ed “(LL61) Suearig yo Aoyreg ‘repreg onb tsure “(gc61) 
syre 9 uosuey ‘o[duoxo red ‘sdura3uo] elap e A [I aUOU 
JUO | SUWIOD “INIMIIAIOJUL _P J9JJ9,] B oNofNs ‘Issne a[9 Iso 
uonsonb oun eg ssuodgi-uou eT “WUasoLjUI StI nb ssuuossod 
So] JUoUUOp onb sesuodgl so] JUSOUSNITJUI SINOMATAIOIUL SOT 
‘S9QUUOP OP «SaI]NIU» SINB}D9T[09 Sop 3199,p nar] ne,nb sa 
sopni9 soo op [edroutid ourey) a] “(6861 SPAIN) saygnbus 
S93] SUP SINSOUI Op INdLO,p sdInOs onb jue} US INOMOIIA 
-IQUI | op JUd}Tey Inb sajorpe say JOS XNBIqUION “snssado0id 
puodes 99 suep jueyOdUII 9[Q1 UN onof INaMOTAIOIUI | ‘IC 
‘(C66] UeUIpPNS 39 ZIeMYIS) UONeOIUNUTWIOD ap snssad01d 
un Issne stew “uepuodal np JNIUsOD snssad0Id 9] JUSUIaTNes 
uou pucidwiod uonsenb oun e aipuodg: ‘juepuedad 
‘(6IP ‘d ‘6861 SeAOID ‘adurexe sed) yofns 9] Inod 1919} 
-Ul.] 19 (1661 ory 10 MNse] ap ‘xoH ‘ayduuoexe sed) oxas 9] 
‘(Erb-I pb d “6861 SeACID ‘a[dutoxe sed ‘0A) a3¢ J ‘(bL61 
wingpelg 30 uewIpng ‘ajdwioxe Jed) gyLe[OOs op nedAlu 
9] JUOS sajUoUTIEd sonbystig}oRIeO soy ‘soInatIajue sopnyid 
Sop UOTIS “«sed jes dN» osuodaI Be] Bp xIOYO 9] ted opus 
ua onbijdxo suonsonb xne oipuodeal e& jUeISsISUOd 9)08 J 
Op JNtusOd yoodse.| & JoIOOsse jnod uo,] onb sjuepuodg 
sop sonbystia}oese9 say ‘asay} 91199 UOTIS “(1661 Yorusory) 
uonsonb us owg{qoid ne Jgigjut,p nod yuapsosoe Inb sayj90 
39 SoyINsuI nod souuosied sa] ZoYyo JUsUIEJeITNOTed ‘asuod 
-91 91399 jUaSSISIOYS INnb syuepuodg1 op uorodold ey] 13} 
-uowsne j1eJ «sed yes ON» sipuodal ap sir] IqIssod ev] JouUOp 
gJenbe] uojes uoNeAJasqo | e10go01I09 onbioesy) JuowOU 
-UOSIeI 9- ‘JUepUOdEI Np UONeA}OUI ap Ja sadua}9dWI09 op 
‘sgouessreuuod op neaalu np onb Isure ‘ayoe} e] Op SI[NOJJIpP 
Rl WP UONDUOJ sd QI[IOvJ OP UOTINIOS eI IIsIOYD ‘“«a[qrey» 
Jntusod snssad01d un .p 3eI[Ns9I 9] ISO o][a steu ‘opqeideo 
-oe asuodai oun jso «uortutdo sues» asuodal ey] ‘oyewndo 
asuodal e] JouUOp Inod aitesseogu [eJUSU! WOJJo ,| JoAO[dap eB 
9ANOuI sed 3sd,U JUepUOodaI 9] Is aJUaSgId 9s UOTENIIS o00 
“QUIOeJ Op UONNIOS ap SULIOJ ouN jso «sed yes ON» sIpuodel 
onb juatnos yorusory “(Ep ‘d ‘1 R86] Josseig 32 ueUINYIS) 
9sT[QN aj np ainjeu ve] op uoNdUO] 3sa oyIp yuouoIdoId 
uonejuswsne | onb 3a asuodai 93399 yUouUOp Inb syuepuodel 
op uomodoid ey] ayuowSne «uorturdo sues» no «sed yes oN» 
aN un .p uonesin | onb nuuosal JUsUIS|[e.19U9S 1S2 |] 


“SVd LIVS UN” GSNOdHU “7 


“Q[dIUe | IP SUOISN[IUOD Sot 
SUO|[NULIOJ snou ‘UOT}Des sUTATIINY ke] B “UIyU_ “osATeUe | Op 
$}€][NS9I So] SUOJUaSeId snoU 39 sUOSIT{N snou onb se[gpoul 
XNap Sap UONeIOgRIP,| Op SUOINISIP snou ‘sUOT}DeS SUIT} 
-das jo OUIQIXIS xNY “aSA[VUL | OP SA[qLIvA Sa] SUOUasgid 
snou ‘uordes sulginbuto eB] YW ‘suOUIquIOD snou aonb se[gpoul 
SIOAIP Sop JUSWIAAQIIG O11.) UOK|Des aUILeNb eT ‘exe[dus0s 
aINjoNs Ins] Jasiogid op ule ‘[IeJgp US SogUUOP Sa] SUOA 
-LIO9P SNOU ‘UONDSeS sUUISION e| Y ‘Josc[eue | op UOSeJ BI 19 


‘«sed HWS ON» osuodal eI SIIP-R-}sa 9 ‘aJUepusdap s[quLIA 
21}0U Op dINILU LI SUOALIO9P SNoU ‘aJURAINS UOT}DAS ET YW 
“INIMIIAIONUL | 
UOTES Jo JURPUOdSI 9d] UOTIS Sagsse]D JUOS saINSOW soy 
‘Q[QPOU! 99 SUC] “S9SIOIO UOTROIJISSE[O B S[QPOUI UN sUOSsSITq 
-e19 snou ‘asAyTeue SUIgIXNap PI Op Sed a] SUR “SINOMOIA 
-I9}UI So] SUBP SdYOQUUId souUQUI-xnd JUOs INb syuepuodg: say 
SUBP S99IOQUID SOINSOU :XNBIAIU SION) & «a[duIIs» sJapoul 
UN 1So B[QPpOW VT] “9}deJUT aJSso1 onbriyosressty JuoWsInd 
ainjonqs ey] ‘askTeue s1QIuaId ve] ap S10] “sIOj a[nas oun nb 
S9MIIAIOIUL 939 1U0,U INb xnoo sudwoos A ‘sjuepuodel 
Sop J[QUIOSUD,| B SUOSSOIOJUT SNOU snoU ‘a}INSU_ ‘INOMIIA 
-I9JUI UIQUI 9] Jed SIO} xnop saMmoaIArojUI yUOS INb syuep 
-uod91 xne juowenbiun suoyseye snou snou ‘olgrueid vy] 
sueq ‘soscyeue xnop suosiye9l sno ‘axa[duro0o snjd yuataap 
9[Qpoul 9I ‘stng ‘stidosdde afepou np 3a saguuop ap s|duurs 
sinjonyjs ouN.p suoyied snou ‘spore juasgid a] sueq 
‘onbryoseigiy yuowioind sed sa,u neaatu Jed yuouiadnoi3 
QJ Ted ‘agsiolO UONROIJIsse[o RB esepuos op ueyd ap 1dr 
suolopied snoN “SINSMIIAIOIUT So] Jo SJUePUOda So] IIOARS B 
‘SOJOUIISIP UOTJVOIJISSe[D op SoINjons xnop suep sagdnois 
1UL19 SUIUIOD SOINSIUW SI] ITOAVOUOD INeJ [I ‘aINjONNs 91199 
op ajdur09 11u9} Inog ‘seguUOp sap axatdutoo snjd aimjonns 
oun e nol JouUuOp inod onbryoresaty yuowoind yuowiednoi3 
3] }Wdwousjut Inb uoNengs ‘soy argmueid ey onb INaMoIA 
-I9}UI UIQUI 3] Jed SQMOTAIOIUI 939 sed JUO.U sjURpUuodsI 
sop uednyd ey ‘afaA9 ouretxnoap 9] yueIng “aygnbua.p sajoho 
XNOP OP Sa][2d JUSqo|Sue [aued ap sagUUOP SON “SINOMOIA 
-I9JUI Sa] SUBP SaTOQUID yUOS sjURpUOodEI sa] 30 ([eUIPN} 
-IZU0] osepuos op uvd) syuepuodgl so] suep sadgoquia 
yuos (7 9[9A9 39 [ BJA) SoInsou soy] ‘[oued sed ajgnbue 
oun,p sio, sagsod sonbrytjod sized say ins suonsanb op 
aLIgs oun & «sed jes SN» SaSUOdaI Op SIQUIOU J] INS INOMOIA 
-I9UI,p Jo JUepuodgl ap sjajjo so] suoIasATeuR SNON 
‘Inatadns nesalu op stun 
anne sun suep sogdnoidal ang jUsAned sagatasqo sgyluN SIT 
‘sonbryoselaty XNvaAlU op dIquioU pues snjd un .p a}dwI09 
lua} ap youLed syepouwl 9I ‘Isa snyd In? “uonjoag inb sarqeu 
-BAOD SI] JUSUI[IOVJ Zasse I9IF9IUI,p yoULIad jo ‘sogarOsqo 
spun soy sojno} Inod swigu af sed jso,U soinsoul op 
JIQUIOU J] NO XNIO dIIp-e-}S9 9 ‘sgiquinbs uou osepuos op 
sueyd xne sonbrdde s yned onbryssessty a[gpoul a] “soysay 
-IURU SoseyURAR SUIeLIOD aJUSSoId XNBATUN[NU osA[eUR | 
SIVUL ‘«So}TUN Sd] SUBP Sa9IJOQUIO SOINSOWm daAv sepUOS 
ap uejd a Joskjeue.p suoXowl somne.p sIstxo [I (9661 
UIAISPJOD 39 Bue X ‘9661 Siopfius :¢6-L8 “d ‘S661 UlaISpjOH 
‘giduroxa sed ‘II0A) «sagjgd91 SoINsoU» op dIIP-k-1S9,9 
‘sa[eUIpNIIsUO] SeguUUOP ep SATeUL,| IP SIO] ssnenjondy 
JUSWATe39 ISO XNVIAIUN[NU s[Qpow np uOHEsTNN,T 
‘aguiod opueis snjd op suonesi[esougs 
sop & Japgooid op joulIod InemoIAJOIUI | 9p Jo JUepuodgI 
np sanbysig}eivo sop sjojjo soy Jed INaMaIAIOUl | B 39 
juepuodgs ne sanquye souRLeA ve] J90e[duas ap sitiqissod 
eB] ‘Juepuodal np Jo INOMOTAIOIUL,| OP NBSATU Ne SagINsoUt 
soatwordxa sopqeiuea sop sjajjo sat onb sure “InamotAa 
-IQ\UL,| @ 99T] QOULLIVA eT JOWINSS,p JouLIod onbryosess1y 


jaued sed sajanbua sap seo a] SUP INaMaIMAJUL.P SJejo SEP LONESI|SPOW :{P|eASOO7 Je Alex! lz 


"9Q'OE'UIANI[NY"90S ® IP[Sasooy] W9ed ‘aq"ov'udANaNy “90s @ AJoyoId uel : jounoes 


“UINIZ[ag ‘UdANT] OOOE ‘AZ JeeNsUsA URA “q ‘UdANT] Jo AjIsIOATU_) ‘ABOTOINOS Jo JusWUTEd|G “IP[SASOOT] WseEH Jo AloyoIg uel 


Qgpour a ‘WoIOINS oq ‘“aoURLIRAOD eI ap asAleUe 
UN Pp NO 9dULLIVA EL] Op osATeUR UN Pp Sed 9] SUBP Sdg[NULIOJ 
saqjao aonb sajsijeor snjd jyuos xnedAlun[nul sfepoul 
UN Pp USAOU NB INIMOIAIOJUI ,P SJaJJo Sap asATeUR | JUDPUd} 
-snos tinb sasayjodAy say “uanbgsuos seg ‘sanbnsiye}s 
SUdAOUI Sop Jed je] OS SINOMOTAIOJU! XNV SOATLIAI SO[GULIeA 
Sop uonesyess | ‘XneoAlUN[NU sTepow uN sup soINp 
-ONUI JUOS sJURpUOdaI say JURSLIQJORIED SajUoUT}IEd saTqeLeA 
SQ] IS UBDYDE Sed J] ‘UOIZAI 97}99 Op SINIMOIAIOJUI Sop Jed 
SOMIIAIOIUL JUSWI[QRIQUIASIVIA JUOIOS SIQI[NOWIed UOIs9I 
oun.p sjuepuodsi say onbsind sipuojuos as juswe[on} 
-U9A9 JUSANd SINIMOIAIOIUI Sap 39 sJUBpUOda sop sonby 
-SLIQIOVIVD SO] ‘SINOMOIAIOJUI XNe presey ne sg}oojje sed juos 
ou sjuepuodal sat Is “(ZOE “d ‘“p66I XOH) o39[duWIOOUI 3s 
sjuepuodal Sd] 19 SINOMOTAIOJUL SIT JURSLIQ\ORIVD SoTquLIeA 
sop UONesITeuosoYpO || Is aN ayoLIdoId ‘ayepoul np saaned 
-1[]dxo sayquiiea xnv sgUUOPIOgNs JUOS JoULLIRA B] Op sajuesS 
-oduwi0d say anb uolssaiZ91 ap SJUSIOIJJ909 Soy UsIq Issne 
*XNVIAIUN[NUW B[QPOUI UN ,p Svd oJ SUB] ‘“seguUOP sop enbryo 
-JeIgIy dINjONIs e] op, asned v ayIeJsHes sed yso,u ‘sdwia} 
np wednid ey, ‘nb uontpuod ‘sayuepusdgput JuouIeT[on NU 
HOS sjuepuodal sap jo SINOMOIAIOIUI sap sonbuysLg}oeIeS 
sat aonb a3txo sonbysnejs sapoyjeur sone .p uoneoydde J 
"«SINOMOIAIOJUL SO] SUBP SQYSIU sJUBPUOdEI ddAe BBepUOS 
op ued op seo ua gudoidde snjd 9] [No ,] Iso onbiyosess1y 
J[QPOW FT “(p661 XOH) osgnbus .p saguuop so] Ins INIMIIA 
-I9IUL Pp SJajJo Sop osATeur | & XNOMU 9] JUQUUSTAUOD INb xnao 
Issne jUOs sonbryoress1y NO XNedAIUN[NU So[epoul sa] 


(6661) 
Jaysog 39 siopftug 19 (g661) MNVZ] ep 19 Yory (S661) 


I 


UIDISPIODH ‘(7661) YSnquepney 39 YAIg oUIWOD ‘sjonueW 
SIQAIP SUPP SILIDQP JUOS SITQPOW Sa_ “neoatu onbeyo k sagt] 
sofqeuea sop Jed xneoalu xnep xne s[qenquye soueleA 
999 BJQpOU oJ suRp JOOe[dUIOI Op ddJOJJO.S UO “WOIOINS 
OC “SOAP BUD UOTLIVA LI IP 39 Sa[OO9 JUS UOTIVLIVA 
PR] ap ajdulod oipuol op youuod xneoalun[nw s[gpow 
un ‘UOnBONPY Ud BYdIOYEI BI Op sed a] sUep ‘aydwioxs 
Jeg ‘(pee d ‘p66 [eISLUO,] 19 DJaIgIC) Inde | sp sinjonjs 
RP] ap oxafdwoo snyjd juowsjen ne 3Ojn[d stew ‘xneoAru 
SIOAIP SOP SI[QVIIVA SI] OUD UDI] 9] JUeSSTTQLIE aT[QUUOT} 
-OUOJ SULIOJ BT B Sed UOU jUAT} SeTepour sao op odAq guyenb 
Be] ‘nesalu onbeyo ke agid0sse ay]IqeueA ev ap a}dwW0o 
JIud) Issne ste ‘UoWodnoIs op xNedAIU SIOAIP xne yUspuod 
-Sd1109 Inb sa[qeueA soy ainpour yned xnvaAluy[NuW s[epoul 
uQ “xneydoy sop suep sodnois sjuoned say no sasseyo 
sap suep sodnols soag[g so] sWWIOD ‘sgIdap sinoisnid 
B OJIP-B-1S9 9 ‘s9giJOQua saguuOp sep asdyeur | e sudoidde 
[Qno un yso onbryosess1y nO xnedAlUN[NU s[epowl a] 
‘jaued sed ajgnbus oun .p seo a] suep INIMIIA 
-IOJUL.P SJaJJo So] JosATeue Inod xnesaluN[Nul sofepow! op 
uonesI[NN | JaNsN]II,.p 3so spore | op tedrourid yng aq “yed 
anne .p ‘ajduios saguuop ap s[quissud | e 30 ‘ued oun.p 
‘SagUUOp op d]qUIZSUd | Op s[qUIdsUd-snos UN e& suonbrTdde 
snou onb xnedAlun[NU sa[gpoul SJOAIP SUOSI[NN snoN 
‘a8[aq uonendod ev] ap seidne saye10j9e[9-}s0d sajgnbue 
Sop oIped a] suep sguow youed sed ajgnbua,] ap satoAo 
XNOP OP SIO] Sot[[IondeI «sed yes 9NY» sosuodgl op siquiou 
QJ INS INIMOIAIOIUI | Op 19 JUepuodel np sonbysig}oeIe9 
Sop JoJJo,] SuosAyeue snou ‘afore yuasgid a] sueq 


NOILONGOUMLNI ‘T 


«sed 


WS ON» osuodgs ‘xnesatuN[Nu sofapour ‘youed sed saygnbus ‘INdMIAIOIUL,P SJa}jo ‘ogSIOJO UONLOIISseIQ “SATO SLOW 


‘jeued red ajgnbua_p seo ua 


INOMITAIOIUL P SJoJJa say JosATeue Inod ayn po} 39 afdnos sad} [NO UN jSd d9SIOIO UONBOTJISSL]O & XNBIATIUN)[NUI JJ@pou! 9] 
aonb suonjouos snon ‘sasAyeue xnap Sad op sje}[NSgI So] SUOIeduIOD snou ‘sing ‘ouUOSJod oUIQUI Ee] Jed SIO} XNOp SamaIAJO}UL 
939 JUO Inb xnoo0 Ins yuouonbrun ane | 39 sJUepuodyi sa] sno} Ins yuUeyOd oun | ‘saguUOP sop exeTduIOS dINjONI\s 93199 ap 
sajounsip sasAyeue xnop & suopgo0id snoN “a9SIOIO UOTEIIJISSEIO P| :ayeqo]s onbryosesg1y aINjons}s ouN JoyUasoId stoyojno} 
SUBS ‘SINIMIIAIOJUL SI] SUBP S9LOQUID JUOS sJUBPUOdyI so] 39 sJUBPUOd I So] SUP Sd9}JOQUID JUOS SoINSOU So] NO dxaTduUIOD 
dINjoNI}s 9uN 9}[Ns9I Ua [I ‘JUaNbgsuod Jeg “SIO} xNop sa] JuUOSJod oUIQUI B] Jed S9MOTAIOJUI 939 sed JUO ,U SaTOAD xNop xne 
gdtonsed quo inb syuepuodai sap wednyd ey yuowiaynes aygnbua ,p sajoAo xnop ap saguuop sz] JUaUUSIdWOD Joued ap saguUOp 
$99 ‘a3[9q uONe[Ndod e] ap saidne safesojo9]9-3sod sajgnbus sap a1ped 3] suep sat|[ionool joued op saguuop sep e suonbrydde 
snou onb saguuop sep 9iiyenb e] ap Inayeorpul uN suosAyeue SNON “«S99}9d91 SOINSOUI» Sop SIIP-R-1S9,9 ‘soTeUIPNIIZUO] 
saguuop sop JosAyeue Inod Jast[yN,] Issne nod uC ‘ajgnbus ,p saguUOP SI] INS INIMIIAIOUL Pp SJoJJo sop asATeue | & UIIG 
JUSTAUOD XNLDATUNNUI J[QPOUl JT “XNVIAIUTI[NU So;Qpoul op suoNeoidde xnop ap uosteurquiod ek] WWOep spore yuasgid a] 


aWnSaa 


tLATHASOOT LYAAD 3° AMAMOId NVE 


uonvoydde sug :jouvd avd sajgnbuo sap seo a] 
SUBP INIMOIIAIIJUL, P SJIJJO SOP UOILSITIPOJA 


El? 


epeuey enbiysyels 
vec-Elz dd ‘Zz ,uU ‘9Z 1OA 
000z esqweadep ‘ajanbua,p sanbiuyoe | 


"EST-EvI ‘OZ ‘asenbua,p 
sanbiuysa] “UOISNJUOD dade dUISTUBOgUI UN UO[Es asUOdgI-UOU 
aun ,p ayIns Jed juononb sed sagynduit sinayea sap yueuaidui09 
seguuop ep so9i s}gnbue.p suoneuinsa sap sinod sieiq np 
SUONDELIOD “(h661) “A~'D “TWANUYS 3° “H ‘Ad “A SLYNOONVU 

"SSI 
oMMapesy OX MON “P8I-Ebl “7 ‘sarydessorgqrg pue Arooy], 
‘Cspg “urqny “g’d 19 ‘Mope ‘DMA “UPIIO I) ‘sdaaung ajdung 
ul DDG ajajduoouy sueq ‘asuodsal-uou jun Joy jUOUNSN[pe 
sunYysIon “El JaideyD “(€861) ‘fd ‘NFUNGHOS 1 “TH ‘HO 

‘LI-I ‘TI ‘ajenbua, p sanbiuysay ‘soyuenbuew a\gnbus .p 
SaguuOop sap JUsUO}IeN) FT (9861) ‘d “MAZUdSVM ?9 “OD ‘NOLTIVY 
“6TS-LIS ‘Ip ‘Uuoupiz0ssp jooysyojs 
updidaup ayj fo jousnoy ‘sKadins oduies ut asuodsas-uou 
Jo wajqoid ey (961) ‘N ‘M “ZLIMUDH ? “H “W ‘NaSNVH 
"SuOS 2 ADTIAA UYOL :yIOX 
MON “WONIPS .¢ “sanbiuyoay 3uyduygy (L161) ‘O'M ‘NVYHOOO 


aIHdVeaOOITsla 


“Q7oIe 91}0U JOIOI[IUIL 
& sgpre quo snou Inb saireyUsWIWIOD sino] ap sjulofpe Joyo 
Ud SINDJIVPPI Sd] 39 SINDJVUIWULXS $3] SUOTOIOUIOI SNON 


SINGING YAN 


‘uoryeyndut 
Jed tu uonesgpuod ey op juowiajsnf{ei red iasw09 ynod 
au uo] onb asuodgi-uou ey] & ap JUeLOdUIT steIq UN 9)STx9O 
IL.S 9[9N ong Weunod spoyeW BON “ZWMIN 19 UosUeY 
9p ANIeUINSd | onb sj}eI[NsaI SINIT[IOU JUDUID}OU Op JUOUUOP 
SINDJCUINSS SOU ‘aT[Ie] spuLIS ap UOT[HUeYdE UN.p soidne 
DIVI[IXN I[QUIUA L] INS S9QUUOP SI] INOD Z[qIeJ & IN][IoNdeI 
nod uo | anb ja orrerjixne sfqeiea ey] 39 afediouud sfqeuea 
BR] oNUS s9D0UO0UOId SIIeQUIT UONe[OI OUN 9)SIXO [I.S 
‘ZUIMIN 19 UssueH op INsjJeUINsd | 
OP I[90 B sINayeUINSa sou ap soUPULIOJIOd eT JUoWIONbUTduIS 
39 JUoWANbIIOaY} suOIedUIOD snou sind ‘sarreyUN $}NO0d ap 
guUUOp e[quasus UN Inod UOT[HURYDE ,_p sofeUINdo saT]Ie} soy 
SuO[NITeD SNON “(9P6[) ZIIMINH 39 UssUR] Op VpoOyIeul P| 
R JUSWISULIOJUOD ‘sjuepuodal-uOU sap sseUUOT[HUeYd9-sNOs 


Jed ssuodgi-uou ef & np jueoduN juoWT[anjUaAg sIeIq 
9] SUOUTUITS SNON “asrerIxne o[qeuea el op uoNe[ndod ap 
auuaou | sed yeuuos ou uO puenb oyediound s]quuea BY] 
ap yo{ns ne asuodgs-uou ek] op ajdu109 Tua} Inod saseyd xnap 
B asVUUOT[HULYSS,.P SpoysuI oUN INS s9puoJ UOISSaIZ9I 
Jed no juononb sed sinayeuinsa sap suosodoid sno 


SNOISNTONOO °S 


orl €SZ OL OG AS Gere 14 £0 

vel 697 vL EVA Cee SSal G £0 

61° €CS £8 SOG ASU cer v TO 

[eb £6 €9S 68 alle fol) AXE G 10 
“f 

Coat VET €L Ne Sil wee v £0 

eT I OST 8L or‘ EG BST é £0 

161 LLY C8 NG eS AS6r6 v ime) 

C81 vis (G0) 9r'T Lél BST 6 10 
«P 

aUNOMI 4 te cy ee ey 8S ys) 


(L= '9 ‘S0=9 ‘007=.9) .4 
e yoddel red (719 ,p op sAneyal oy0RONyq 


Z neayqey 


101 
sed suojuasgid ou snou onb enburduis uosrtereduios anne 
QUN SULITJUOD 9] 39 Wpaid a] UIUOD “WesedsIp |p Ins ,J Op 
a8uURAL | ‘OUIZLIO | Jed assed UOISsaIS9I Op OHOIP PI IS 

‘sIndyeuUINsa xnep se] Inod aurgui a] JUsWoNnbHeId 
yso °y yeurndo asJoAUl oseuUOT[NUeYSE,p xn} 9] ‘UIyUg 
‘aseyd osgmuoid op uolnueyse,, & osejURAep JoNOT[e,p 
gqissod sa [t,nb uarq Is ‘oseyd owigixnep op uo][nueYyse 
yned snjd un seae stood snjd uorsseis91 sed Inoyeuinsa 
un i1uajqo yned uo,nb sored ‘asIoAUI,] SUGAIOSqO snou 
‘°u ‘aseyd auigIxnep op uol[HuUeYyss.p sjeundo ate) ey] 
Inog *,p inayeumnsa,j nod onb ,7 1nod opuess snyd ysa ‘7u 
‘uoTTNURYOE ,p afeuNdo ofeNtur o][1e} ey aonb Issnev suoye}suOd 
snoN °,p Juanonb sed smayeuyse,, aonb sovoyyjo snd 
489 ,J UOISsaIZaI Jed InayeuNsa J ‘UoNe[ndod 91329 ap sed 2] 
sueq ‘oulsu0 | sed assed ou uoIssoi39I 9p a}10Ip Be] anb yp 


uoisses6ei sed no juanonb sed uolewijsa,| nod seseyd xnap e eHheuuo|jWUeYOR :887 2 JOJeEXO Zid 


jusWarNe ‘g op JUsUIO}OU oJQTJIP Y onb suojON “7 nes]qei 
ne soajuasaid yuOS 7 19 ,P OP SOATRIAI SOOVOIJJO SoT 


‘ze'se9 = ““s 38 co‘osel = “s 
‘eg'ech = “9 ‘go'rarl = “s ‘ze'9zHz = As 
‘pS‘99L = 5 ‘ZOE = “N ‘O00I = N ‘E80 = “J 


‘Lol = “d ‘g8'l = w‘se'0= 9 ‘ZS = 9 ‘761 = ¥ 
:SJURAINS So] 
e uoddel sed ,1 10 


N 


juos uoye[ndod ey op saneguiesed saq * 


_P SINSEUIISS SOP SATJL[AI SNOeITJJO ,| Joredwiod Anod jusure] 
-[olorje ogigues uone[ndod oun p suoasas snou snoy 


sgsodoid 
Sina}Vultjss sap snbisuiduis uostvreduoy = ¢'p 


I 
<yg- dz 


Is ,4 onb aovoryya snjd iso .7 Ip yuowoNNY *,7 10 ,4 aNua 
SIIe[MWUIs UosTereduiod oun eB Jopgo0id suoAnod snoN 
‘K 12 X OP UONLI9LIOO op JUSIOIJJO09 9] ajUasgideal d no je 


HS 
Meee ceo I 
EST - YM 

a HH2 o a 
ge SO aan, MSG SLO 

E51 - a) a °u ‘ é 
no 

: (6 x AHH xT a Z ug 

(op) \(Y- “dt) O4-C""O8- G+ aaa 

‘9-1 

x <yy - JZ 


I 


qu199,s ,4 onb stogid snjd qos ,p anb yuepnoa uONTpUOd eT 
sloye “(¢°¢) red saguuop ,u ja u ‘y ap sayeuNdo sineyeA soy 
DAR (,P)A OP 2I90 B UOISsaIdxe 9y399 sUOIedUIOD snou IS 


HH?,, 
> Ufa ee fe 
(Sp) 2? (y — BH ye 
HH? 
(le aad 
BEG 


JUSTASP ZIMINFT 
-UdsUe]{ Op INayeUINSE,| op speudo soueLIeA ve] ‘sIOTY 


(p'b) CM? + yz HHO 
ITER 
(5m - ts) 


HE Gp Cael 
¥IM ial I 


se) 
70) 


“INS DUIS ‘¢ UOTIDES PT B SesTTNN 
g]joo aonb (e8ueise] op Inajeoydninu ey] ‘ol1p-e-3sa,9) 
onbruyoe) ourgui ev] UoTas (Z'¢) sed guUOp INja0 eB a[qe{quias 


; 4 
u 
‘M9 


‘nagid ynoo un Inod uo][HueYyse .p sfeudo 
oy[tey ey Jajnoyeo suoAnod snou ‘a4 inod (saseyd xneap 
B oseUuUOT[HUeYIE,| B INOSeI sues) o[duIIs sIOJegTe UOT 
-ueyde UN suOUUONDeTIS Snou Is ‘,f INdJeUII}SE | INog 

"€ UOIDAS BT B SQUUOP NOD op UOTDUOJ e] Op JULAIAS 
snou ud (,4) ZyIMIN}-ussueH op Inoyeunsa | e sasodoid 
SINIJVUIT}SA so] Joredwiod jueUdJUTeU sUOTTe sNnoN 


sa'0+2 |e 9 


}N0d np a3du109 juvua3 Uy «Zp 


“S9SNIINOBLI SUIOU SUOTIPUOD 
sop suep , 4 aonb aovorjjo snjd ang ynod .7 no _p ‘sajuesty 
-JNS JUOS IDI SUOINISIP sNoU JUOP SUONIPUOD saz sWIWIO|D 
0. <g doar ‘zg <°¢ Is USWUIE}OU dAJOSGO,s UONIpUOD 
a9 “sAntsod jso (Ep) BOUDIOJJIP PT IS ZMINA-ussueH 
op InajJeWINsa | anb sovorjjo snjd ysa ,7 ‘juanbgsuod Ieg 


(J - dz) 2d 


(ep) a- 0M 

i ju u 

ee es Cd 

oS E aytel 

SUOAR SNOU ‘sINaTIIe eg 
Zo) = | no “Yz>y 1 Jz>w Is (,¢ onb aovoyyo 
snd yso .p ‘WIP uoueNNe) satIsod ysa soUdIOIJIP eT 
Al eve AxZ u 

wr) (sea-“sud} + 


4 ie iphone 
(‘Sew - “swd|=- 7 |=CP - (OA 
(ZZ) suep SUTUIOD IuTJap 3so ,4 NO 


000Z aiqwesap ‘ajenbua,p senbiuyoe | 


u N u 
(I'y) Steen ty fe - =|. (QA 


cee yy 
Jed sguuop 
1SO ZIIMIN}-UssUeH] OP INIeUINSS,| op soURLIeA eT 


ynod np a3dwi0d 119} sues [ "fp 
‘1d-In[99 ap o}duwI09 jueUD} Ua snd ‘Nod Np a1duI09 
Jtua} sues ploge.p ‘(Op~61) ZIMINY 30 UISURY Op sina} 


-BUIN}SS Sop a][29 B sgsodoid sinayeUT|sa sop sourWLIOJIod 
BR] JUsUIONDUOSY) suoJeduOd snou ‘UOTOes ajUasgid EB] Y 


SUNALVWILSA SAG NOSIVUVdNOD ‘+ 


ISS" OL SOLIS GC coe cl! TY G v IT s0 s‘0 002 
SEG OS OLIN 8hc Foo FO. C (6 v IT sO s‘0 002 
sce LO 661 USI EEE €O if v T SO S‘0 002 
SCO SL 661 CST Ces, SO. TC I v LE 5Si0) 150) 002 
TLE O08 OST 89'I LOT €0 (4 C T sO S‘0 002 
GSZ 96 9ST 89° LOI €0 Z 6 6 T so sO 002 
99€ TL OL'I 801 LOKI €O F if G IT sO SO 002 
OSt S8 9L'I 801 LOT €0 Z if (G b SO SO Cie 
€SS_S8 OL L8S? tf 0 + G v ee SOMO O0G 
I9¢ 96) OL TASEG EGE €0.. 7c C v ees OM OMEOOG 
USS Ab ASTIN TASES TS ay if v SO) to Wore 
TAS ARINC AS CASSIS) 8 Seg T v TS) Ome TAOMROOG 
OGSTE GUIS ROIS CON vy Co 7%) | S010) 00% 
68€ POi 9ST 891 LOKI £0 Z C C eS On OnOOG 
08s 18 SLT 80°T LOI €0 F if c beeeSi0l 1 0F 0072 
TBE 76 OL 801 LOI £0 Z% if G SO) L000 
me Oe Ry ok a 1S Cae ee lara rs 
u pue ‘°u<°y ap sfeundo inaye A, 
[T nesjqey 


“sguuop sozjeuered sap 
Inod °u ya °u ‘°y ap sareumndo singyea soy aNUOW | nes{qe) 
YJ] ‘UsWaANoedsar aS re) ES sed mS yo “g oguuonuoUl 
-sns (¢°¢) uorssardxa | suep juowaduis suode;duros 
snou ‘(,J)4 op INeIeA ek] WinuTTUTU Ne yUaSINpe1 IND ,u 30 
u ‘¥ Op Sefeutido sIne[eA so] ItUd}qQo INog *,D ‘nagid NOD 
ne sajfouuorsodoid juos /u ya Cu ap sapeuindo sinajea so] 


tee Se h,2 
. ten aaa ie) 00 i sa 


Oe Men 1h 9), 20) 
g/(T - nen + 1.9 


“(Zy - ga = 


(p'€) 


SuOUd}GO SNoU sIOye ‘ Sie ses 
10 Se s=@Q ‘“ly'o+o)/% = 4 onb suosoddns shou 1s 


Dp) + 2(Js- ‘yh = 


a motos 
ey co M =) 


sdZ ONE li ae 
gol YM + SV 


(m'2 + 9) Zs 
(s M ~ is} 


:JUOS SonuUd}Qo IsuIe sayeudo sinafeA say 
"OD ext nagud ynoo un Inod ,p op soueiea P| WiNuITUT ne 
quosinpgl Inb ,u jo u ‘y ap sayeuitjdo singyea say Japnoyesel 
Inod a8uerise’'y] op Inayeoljdyjnu np suOAIJES Snou snON 


EAL Oso eu, th chee 


4 
Ge) a bl aes woeGeye 


Jed guuop jso nagid ynoo gq ‘uoTyestUTUTU Be] Inod nagid 
1N09 Np as as uO ‘saguUOp ap 9}09][09 oIQMWUIOId BY Wey 
sed eu uo nb que} 'w op angyea eT sed yeuuoo ou uo nbsing 


x95) 


ap ““n uolmueyog-snos ne g1oosse alrequN yN0d & 


2 


‘'n ap soidne aD9]]09 ap 
WOIjJo Jorueid np s1o] sjuepuodgi soy sed stusnoy A 
INS S]JUSWISUSTasSUaI Sap JUSWOTe] Np sJreyUN INO 


‘e ‘oseyd owiarxnop 
ap uoT[HURYOE | op seidne 4 Ins syusWIOUSIESUAI Op 
Q99[[09 ap WOsjo Joruaid nv g190sse oseWUN QOD =: 9 
‘po ‘aseyd 
aigtueid op uoT[HUeYS,| B gt9Osse oMTeUN OD =: {2 


‘JINS SWILUOD SIUTJEp SaITeIUN s}NOD sej JUSJUSSgIdaI 9 say NO 


uo + ‘yu! 


(1’¢) 


red aguUOp 1009 op UOT}DUO} RE] ,p Inod suoJgpIsuoD 


2+ Ud + 


{4,9 =) 


uoissei6es sed no juaonb sed uonewNsa,| nod seseyd xnep e aHheuuo||jUeYOI :897 Jo JOJeExO OL? 


“aigijnonsed souewea oun Inod joo 3] WiNWTUTW ne 
juasinpai Inb no Jerjnonsed ynoo un Inod sgsodoid sinayeur 
-1]S9 Sop SoUeLIeA P| WINUITUTUL Ne JUasINpel Inb ,u 3a u ‘y Op 
soyeuNdo sIng[eA So] JOUTULIDJ9p JURUDJUTEU SUOTTe SNON 


AOVNNOTILLINVHOd.d 
SNOILOVUA SAG XIOHD  *€ 


(E12) OS Co eked eS SS 
“s 97 - me S.4+ 49 RY = 


JuesT[N us (, J)4 9p INo|[IowW 
nod un Inayeulysa uN suouajgo snou ‘(1°Z) nod swW0D 


(212) CE- ix) g- 
Cova ly pa SG 
9 (4-4) Bier Goacea AS 
! ! 1 Ip t= 
{UC Ra + UE 3 
no 
Wie eee 
a5 4) es (1-4) 
Wate ike ate 
Mle) Genin Whig Gai) eek waded L 
is( 4 -£)+45(4 ae | J=cns 
:oqUBAINS 


g[nuo; ev] sonbijdde suoanod snou ‘(,7)4 JowINsa Inog 
‘d 


Jed y juesedural ua (p'Z) squde .p sgynoyeo yuos cs ie) us no 


Uu 


17 
S 
(Or'z) she Wine 
F u 
Js 


Ge 


Jed ee So J Op datNeUTxoidde 
SOURLIvA eT BUSMIOANSodso é is 19 § 9p sasteiq uou 
SINSCUINSS Sop IUOS _ *s 19 ed anb JONUOU! Sp I[19eJ 189 I] 


+ 
+ is(£- 


u 
; *). CDA 


(6°) 


AXE eX 
se tale 


602 


no « *s/ s = J one 
9 QuIOs JoINJeU — at stew ‘.g inod suonnyos sindisnyd 
Joystxo yreuinod I] * 7S e g ap Inayeurnse un jso J no 


£-2)§+ fo 


(8°) 
‘INS SUILUOD UOISSaISaI Jed IndjewWINsd ,| SUOSSTUTJOP SNON 


soseyd xnap & 
ISVUUOT[IULYII IDAL UOISSI1391 1Ud ANIJVWUISY  ¢°7 


‘stogid snjd yuonbgsuos sed 
jUOS 39 Spuei3 snjd suoT[MUeYdE Sap INS sepuoj JUOS as re) 
2/8 SIhayeUINss sa] anbsind ‘(¢°Z) inayeuINsa | ap af[a0 onb 
gIqQIej snid USWA] Qe[qUIDSIeIJA BIOS BSULYIOI Op INIJeUIT}SS 
309 Op soUeLIRA BT '(6'Z) SuBp aonb sULIOJ OUIQUI FB] & es no 42 


if 7 /u 
‘ ! 2. x 
UX-'x) °K = as 
no ‘(¢°Z) suep ‘juauaAtoodsol 


re 
sabi re Soe Miles =e 


(L'Z) z Zot HED 


jo Se aise Fs ae aS 
:alIp-e-3sa 9 ‘2 og qed 4 7S ques 
-ejduial ue (,p)4 op InoyeurT}so ane uN suouaigo sno N 
ree 
op rojdwa | roynsnt Inod jusuINsIe SUIQUI NP IIAIOS snou 
suoAnod snoy ° Ss OP JUIBIBAUOD INIJeUIT}S9 UN 3sd Yy Jed 
_4 quedse[duias us Si saide ,p anuajqo uorssaidxa : onbsind 
iS Jowijsa nod Ks Jasl]un.p jainjeu s][quiss [J ° ‘S op 
Ssteiq uou inayeumnse un so “ aS onb Jaustjnos ap JUaIAUOD I 


soya 1¢ mice z MZ 
(9°) A aie Umea Crea 


3 {2((4-'4) ORs RO iis4 ke 


ut vam Wa ie 
{Sts( ams , du J Ceres <}— =o6 


v 


0002 aiqwec~ap ‘ajenbusa,p sanbiuyoe | 


ho 


N 


MZ a d ju ou a N |u 
($2) Pea gee if ie | tes: I = I =(,p)a 


Jed 

.P 2p sateultxoidde souelea ev] Jouse suoanod sno 
‘sjuepuodgi-uou op uoneindod 
RB] Inod 39 uoNeindod ev] op s[quissua,, Inod aoueLeAod 
RB] ‘UoWsATOedser “UajUasgidar “SG yo “Gg ‘syUepuodel 
-UOU op s}eIJs e| Inod yo UoNe[ndod ev] ap s{quiasus J nod 
N JQLLULA LI Op SoULLIRA P| “UOWIAATOOdsoI ‘“JUa}UaSoIdaI 
as ote) rs ‘X &@ J 9p uoneindod op yodde 9 auasgider y 


(2) Deer rae ts = 8 
oD SU Gaee Sight Sie. S 
no 
AZ e 
: ty + 
(€°7) anny 


Heda eblatend Ne eau 
(t-te o(-“ ea 


Jed aguuop jsa ‘10[Ae], op alias ed uonesiesuly 
OP Spoy}oW LI UOTES sg[NI[ed ‘,p Op DOULA P] OP aIpPIO 
Jonueid ap uonewrxoidde | ‘uo[[yueyse puis un Inog 
(x? (,u/ 1) = ,X ‘otduraxe 
Jed) swiud ousis un.p sanbieul yuos ,0 eseyd aigmueid op 
uo][HULYD9 | sede _p sag{nopeo sayjao ya ‘('n "YZ (uy) = 'n 
‘giduroxo sed) ‘«[» eoIpul,| ap soonbieu 4uos 
'p saude ,p sagnoqeo soyfao $(’n -¢ (u/ 1) = ““n ‘apduraxo sed) 
«UZ» SdIPUL | op ssgnbseur yuos ““n saide .p sag[noyeo uoyTA 
-ueyoe,p sonbysneys soy *z no p= f ‘us‘u='n 19 N/'N='M 
SUOSSIUIJap snou ‘a[eIgUas II31 RI eB JWOWULIOJUOD 


I, 1 


Ai eats eet Mit A= 1 


(77) 
red sguuop juos tnb z}1minfy-ussuey op 
4 X op sinayeumnsa saj ‘yuaWaANoedsal ‘juos 4 32 .X NO 


26 


x= 


hak 


(1°@) Xd =X 
:JINs sUTUIOD saseyd xnap e aseUUOTIN 
-ueyoe dace Juononb Jed InoyeuINsa | SUOSSTUIJaP SNON 


saseyd xnop & 
asvuUo]yULYyds I9Au JUDIOND sed AnajJeWSy 7°7 


“sayuepuodsa0d saynosnu 

-IWI Saya] soy sed ‘uoy[nueyse ,p sanbysneys sa] 19 ‘sonboois 

Soe] Sot Jnes ‘sa_nosnfeu same] sop sed sgyuosgidal 
quos uoneindod ap saneuresed soy ‘o[eigues o[se1 UG 

ae Jed suojuasgidai snou onb s ajquiesus un 

1So soquN sap a[quiasua | ins owwos eT *““ sed guuasgider 


vias “p ap uoy[uRYyd9-snos a] “pn = “Dv 19 'yUo='o 
anb suosod ya ‘juaweansedsal ‘ v 39 ,v sed aseyd ouarxnop 
ep 30 Jomueid op suol[Mueyse so] suojuassidoy “ouISTA 
OWIQIXNep P| ap so] JUopuodai Inb stew ‘aseuuoynueYyoe,p 
odey9 oUIQIxnep BY ¥ SISTA oIQtuIaId ev] op sIO] sed juspuodal 
ou inb soyun “y op (“py sed ogquasaidar) ayens oun 
yo odeq9 OUIgIXNap L] B OYISIA oJQtWOId eB] op S10] JUspUodel 
nb soyun ‘wy ap ('p sed agquasaidaz)ayens oun :sayens 
xnop Ud 9SIAIP Jso (Pf Jed gyuasgidar) uoneindod ey] ap 
g[quiasus | onb suosoddns snou ‘j1ns nb gsodxa,]| sueq 
‘aseUusUI Np sasuadep say aura0U09 Inb 90 
ud asuod9J-UOU oUN B ddRj OIL] STeUI ‘sasvUDUI Sop dist] P| 
OP JWSUISSI[GuI9 | JULINP II[MWUeJ eV] Op oTITe} BT Ins sjo;duro9 
SJUDWIOUSIOSUAI Sop I1U9}Q0 jeLINOd UC ‘saTerTUIe] sasusdep 
Sop ‘SUOSIP ‘UOTeUITSO | Inod oIrel[Ixne s]qeLeA sUIUIOD 
QSLUSUI Np II [Ie} LV] Op Las 9s UO] NO sasvUgUI-s}gnbus oun 
B Jonbrdde_s yuaresinod suontpuod sag *sayno} yuapuodel 
SOUUN W SO ‘19-SIOJ OAD “| <y¥ ‘y/*U = wW NO ‘y BBeUUOTIN 
-UYD9,P SSIOAUI XNe} Ne soyUN w op astwal sues a[dunts 
aoyeg[e UOT[MURYSE UN suOUUONDeTES snou ‘sjuRpuodsal 
-uou ‘u sap anied y ‘odei9 ourarxnap ev] ap slo] azpuod 
-91 ap juasnyai soytun “w onb ya 4 ins syuoulouStasuar sop 
yuasstumnoy sayiun 'w onb suosod sie “odeuuoNUeYDE ,p 
odei9 sigmoid ey] & x oTelIxne a[QeLIeA eI INs sjUaUIEUSIES 
-UdI SOP Sj]NO} JUSSSTUINOJ SoUN ,u say, onb suosoddng 
‘(Op6[) ZIMIN} 30 UssuUeHL 
op spoujeur ey Jed asuodgi-uou ey] inod sg8t09 ‘seseyd 
XNOp B IseUUOT[HULYD9 DdAv UOISSaIsaI Jed no JusHonb red 
ZX Op SiInaeulljsa sop SIp-e-3s9 9 “(pie -d ‘L16]) ueryoog 
qed sasodoid sinoyeunsa sap saseyd xnap e sseuuo][HUeYse 
B UOISIOA OUN JUONIT}SUOD [or suosodold snou onb sinoyeur 
-1]S9 Sa] “UOWaTe]USWepUO,y ‘soseyd xnap eB oseUUOT[HURYDE 
UN_,p Sd J] SULP Z}IMIN 39 UdSURHY Op oseuUOT[HUeYDE-snos 
Np s9p!l,| SUOAINsInod snou ‘juanbgsuod Jeg “afqeireynos 
one weunod sjuepuodgi-uou sap sseuuo][MUeYo9-snos 
J] ‘JURSYOE SVD FT “SOSTRIQ JUSWASNALI9s JUDIOS s}e][NSOI 
so] onb jurod ne syuepuodg sop jualgyjip syuepuodgi 
-UOU Saj ‘agIpmig enbnsig}oeIeD ev] dUIdDU0D Inb 99 ua nb 
uatgq Oj WeLINOd as [I “Uepusdad ‘sasuadap ap 19 SUOJJo _p 
snjd dnooneag jusUWSTTanjIGeY s}Issso9gU NVOANOU op ju} 
-98]U09 Sa] US sjJuepuodel-uOU sap sgidne syuenbuew s}uSU 
-9UBIASUAI $9] II][fonoeI onb sored ‘apuodes ey anb asno}no9 
suloul dnooneag 1so uoNdo a1gmueld vy] “xno D9AR 49R}UO0D 
aipuaidal jo sjuepuodal-uou op UOT] MURYD9g-sNos UN JoUUOT} 
-da[9s no sjuepuodgl soy Ins jusWenbiun gpuoj InsyeurTs9 
un Jast{4n suoanod snou ‘aseyd oulgixnep op uoT[HURYD9 | 
suep asuodai-uou ap seo use “UepusdsD ‘uO][NURYDE 399 
suep osuodgs-uou op sed e A,U [I,S oseyd oulgIxnep op UOT 
-ueyo9,| saide ,p sonuajqo juos x 30 A no jo ‘sgutun ,u Inod 
ag[noyeo uoTTHURYIE,.p sUUeKOU RI SO ,X NO ‘,x(X/4) = ai 
yso 4 op ouuskour ey op juononb sed inoyewnss,T 
‘uoT[NULYD9 199 Ins { anbysug}oeIed L] SUOINSOU SNOU Jo ,U 
op med e u are} oied snjd op aseyd swgixnep ap uo] 
-uvyos un YSSVy_q Jed suouuonossjg9s snou ‘sing ‘(YSsVa) 
astwial sues o[duris oloyegie oseuUOT[HUeYo? Jed uoNey 
-ndod ej ap ssuun vy ap med e ,u ore) spueis op aseyd 


uoissei6as sed no jualjonb sed uoyewljsa,| nod seseud xnap e ebeuuoj|jUeYOR :887] Je JOfEHO 802 


“W'S'N ‘OS80Z ‘PuRlAreW ‘aT]IAxOY ‘Preagjnog yoreasoy 


OSOL ‘JeISOAA JURUDJUTeEUN ‘epeURD onbysHe}g JUoUIOUUDTIOUe ‘oa] YIYSUNAP ‘eUASIN ‘expNSN ‘eUasIN Jo Ayissoatuy) ‘sonsners Jo ydaq “JOjeyO ‘D ueiqej : 


aigtuieid ap uoT[HURYDSE UN sUOUUONODeT9S sNoU ‘orTeT]IxNe 
a[qeuueA ve] op YX uoNe[ndod op suusAow P| JouINse Inog 


XNvIIUIS SJUIWIIUSIOSUDY = [°Z 


SUSVHd xXndad V 
AOVNNOTILINVHOd OFAV NOISSHAD Aa 
UVd NO LNALLONO UVd SUNALVWILSA ‘7 


‘ZUIMING 39 UssURH]Y Op INS}eUINSE | 
e sgsodoid sinayeuinss sdj yuowonbuidure ya juswenb 
-L109y) suoIedu09 snou ‘uljUq ‘sUTULIAJepeid 3N0d UN Inod 
‘SINDJBWINSS SIAIP se] Inod eseuuo][HueYse,p aeundo 
UOIRIJ BI SUOTNITVD SNON “s9IpNja [qeLRA ke] INs soures 
-S999U SJUSWIOUSIOSUdI saz IIUd}GgO Inod aseyd suleIxnep 
Rl op UOT[HURYDE,| Op Jo oITeTIXne [QuLeA Le] Op UONe] 
-ndod ap uonnqiisip e] JowiNse Inod aseyd argtwiaid ey ap 
UO][HULYOP,| OP SUOAIOS SNOU SNON “SOITETIXNe SITQeLIeA Sd] 
JInod uonejndod ve] ap uonnqmsip ap Io] ey sed yeuuod ou 
uo | onb stew ‘uoleWIyse ouN,p UOIsIdgId ek] JaJOI[gUIe Inod 
SOIIVI[IXNG S9QUUOP op IIAJOS as ynvj [I ,Nbs1o] saseyd xnop 
B eseUUO][HUeYE | & dINODAI UO ‘[eI9UaB3 U_ ‘seseyd xnop 
B odeUuUOT[HULYDE | INS sxe UOISsaI39I Jed no yuaTonb ed 
SINDJCUIT]SS SO] SUOUTLULXS SNOU ‘a}JURAINS UOTIIES eT Y 
‘Toyo 
UOJ 19}N09 yWeunod tnb 9]e}0) asuodai ev] & NODE sues 
ouWg{qoid 9] sIpNosel ap a]QeIA USAOU! UN dIJJO spoyioul 
RI ‘JueyoduII jso stelq 9] Is ‘suIOWURAN ‘s}JUepUodgI-UOU sap 
aseuUO][UeYd9-sNos 2] a8Ixa,nb osrejuoUg|ddns jreaey np 
esned & 9Sn9}NOd sny{d jsa aa sIeul “InejJap 99 sed ajuasgid 
OU Z}IMIN} 39 UssUeH op sseuUOT[HULYd9-sNos op spou}eu 
RT ‘UOTENIIs 93399 JUDWAT[oITed JOBLUOD & IssneJ 1UO (661) 
' [epuses jo oo] ‘yNoOoURY ‘sgIpPNi9 s[GeLIeA e| op puodep 
esuodgl op gitjiqeqoid ve] aonb suas 99 ua ‘asuodgi-uoU Pe] 
Op UOISNJUOD BK [IS SIRI J] JUSUISIQHUS JOUTUITS ,P I[LOIJJIp 
SO [I ‘oIOINs aq ‘“sdgsTeIq JUOWOWOJ oNg,p juonbsu 


SOJULI[NSOI SUOTJLUIT}SA Sa] ‘QUOLID Iso onbYeyodAY oWIsIU 
-CO9UI 9] IS ‘asuodgi op oUISsTURS9U Ne yUeNb sa_qeusynosul 
sosgyujodAy sop Ins spuoj os spoyyou oy09 “Uepuaday 
‘agsuodaJ-uoU P| & OP SIeIq 9] JOUTUIITS B JUSSIA UOTeINdUIT | 
@ Japgooid inod no uoneispuod ey] Ja}snfer inod sagnbrydde 
Sopoulaul SaT “(986 YAzZidsiey 39 UOIey) UOTenduUT 
Jed no (€g6] UaINaYyIS 39 YO) UONeIgpuoOd YY op yusUaysn{eI 
Jed osuodgi-uou ev] yUsANnOs asuddwios uo ‘anbyeid uq 
(LL61) UeIYIOD ep aI[29 & 
AIIL]MUIS SPOUT 9UN UOTIS SgIPNYE s[qeLIeA e] op oUUSAOU 
Ry] sind ‘azerixne sjqeuea ev] ap suUsdAOUW BF] ploge ,p JoUINSa 
Inod saseyd xnop e aseuUOT[HURYDE .p spoyroU P| SUOSTTNN 
snou ‘juonbgsuoo Jeg ‘astel[Ixne s[qeuea ev] op uoNeindod 
ap suusAoul v] sed yWeuUOd ou UO | aonb suosoddns snou ‘toy 
“OIVI[IXNG B[QeUeA ek] op UONendod op suusXouw vy] Weuuod 
UO ‘aNNO UY ‘ddIPNId I[QULIVA kL] Ins sJUOWIOUSIOSUAI So] 
TUINOJ $ajno} sed jUO ,U Sg}IUN souTeyII\ onb sJoye ‘aseuuoTN 
-ULYO9 ,_P SPUN Sa] $ojNo} op JUOUUSTAOId osTeTIxne s[quuea 
PR] Ins s}UsUIOUBIasUaI saz sjanbso] Inod seIpnje s[qelLeA 
eR] op uonendod ap suusKow ke] op uoIssaiseI Jed no jus} 
-onb sed sinayeunsa say gsodoid & (1/61) UeIyd0_ ‘(976 1) 
ZUMINY 39 UssURH OP spoyJoW ve] Ins yueAndde,s uq 
‘JuoUasgidal sIaIWIap soo enb uone;ndod 
-snos ev Inod uoNneWNse oun IUud}qO.p ue sjuepuodg 
-UOU Op UOT[HURYD9-SNOs UN IOUUOTDETIS B 9}SISUOD SpoyIoUT 
aNeD ‘sielq np ouwlefqoid a] sIpnosal op ulje osuodgl 
-UOU Pe] INod UOTdaI09 ap spoyyeu oun gsodoid yuo (9p61) 
ZUMINF 39 UasuLHY ‘sgsteIq o9,p onbsu uoNeWINsa,]| seo 
Joenbne ‘sjuepuodgi-uou sap sal[eo op JUSIQJJIP sJuUepuodel 
sop sonbnstig}oese9 say Is nopns ‘asnodwio0y og jnod sajgtd 
-WIOdUI saguUOp sap saide .p ag[noyed UOTJeUITSa oUN ‘IO 
‘sjaddei sinoisnjd saide ourgur ‘aseuuo][nueysg .p soqrun so] 
$91N0} op saidne sjUdWIOUBIaSUaI $3] JI][IondeI e sed yIssnes 
ou UO ‘sauUOsJad sap juesIA sajgnbua ,p sJo] “JUSANOS SAI], 


NOILONGOULINI  T 


‘oseuuo0][HueYyoe .p afeuindo uonoey ‘ajgnbua,p nod ‘zjIMIN}] 19 UssURH op INo}euINsS| :SaTO SLOW 


‘(9p61) ZIMIN} 19 UasueH] Jed 


gsodoid Inojeult}sa | 9p a][a9 & sINdJeUINSd sod op sdUeULIOJIOd e] Issne oreduloo A uO ‘astel[rxne s[qeueA e] op uoNe[ndod 
ap ouusXoul eT sed yJeuuod ou UO | NO sed xne safqeor{dde (soseyd xnap ke) a[qnop s8euUOT[HUeYE UN INs Saxe UOISSOIS9I 
Jed no juanonb sed sinoyewisa sureyao yep spore juasgid a] ‘olrerfrxne s]qeueA ve] op uonendod ap suueXour be] WeuUOD 
uo | onb juesoddns ua syuepuodal-uou soa] JgUUOT[MURYI9-SNOs F JULISISUOD (9pG]) ZIIMINY Jo UssULH Op 9poyj9u ek] Ns 
sopuoj uoneindod op suuaXouw be] ap uoIssas39I Jed no juanonb sed sinayeutjsa sureyao gsodoid e (pig ‘d ‘ZL6]) ueIyood 


aWwnsau 


1ad'T MPHSNNAH 3? WOHUVHAO 'O NVIGVA 


sjuBpUu0d9s1-UOU Sap ISEUUOTI[IUBYII-SNOS IDAL UOISSI1591 
ied no juayonb zed uoneunyss | nod saseyd xnop ev sseuu0]yUueYyy 


L02e 


epeuey anbiyjsyejys 
Z12-20z ‘dd ‘Z .U ‘9Z “1OA 
000g a1qwisedep ‘ajanbue,p senbiuyoe | 


CTI-LIT ‘opouny np anbys1njg ajats0g ‘ajanbua, p sapoyjaut 
Sap uoljzas Dj ap jianday ‘s\ysiIom Surjdues Zuisn UOHeUINss vore 
[Tes sokeg Jeoryosesory opnesd (6661) “M'N'f ‘OW 1° “A ‘NOA 


“AOTIAA 
UYOL :YIOX MON ‘soisdyg jooyouayjwop fo suoyouny [vI1ssv]D 


ayy 01 uoyonpoauy Up :suoyouns jo10edg ‘(¥661) ‘WN ‘ANWAL 


“UST TAD ESpuquiey ‘Ae UosuIgoy 
‘WOH ONQng JO amNsuy “Wu sonsneIsolg DYW ‘j2nunyy 
Buyduvg sqqin Burs~) aouasafuy uvisadog ‘60 SONA (9661) 
‘M ‘SXTID 1° “N ‘LSA “'V ‘SWWOHL “d ‘YSLTVHTADSIdS 


‘TIT-661 ‘SZ ‘alanbua,p 
sanbiuyza i, ‘aRpowm un Ins sgpuoj aeuolsel UOTIEUINSS | 
queusjoou0D §sjus091 saisoid sonbjenO'(6661) “SN ‘OVU 


“ILI-E9T ‘$8 “UouDID0SsY 10911811015 
uDdidaup ays fo JDUANOL ‘SIO\eUINSE eae [[eUIS JO JOLIa pasenbs 


uBoul JO UOHeUITSS BY], (0661) “Nf ‘OVA 9 “N'O'N ‘GVSVUad 
"68-18 ‘SZ 
‘ajenbua, p sanbiuysa J ‘NedATUN [NUL SaTQpoul op InJed eB safeuoIsel 
suoneuinss,p uononpold (6661) ‘d ‘LTOH “sd ‘VaNOW 
“SI-I ‘Ep ‘sisdqjoup ajwoiavayyjnpy_fo joudnor “[apout FesuT] DDUeLIA 
JoU9 Wopuel & Jopun sJojotposd paserqun Jeouly Jsoq [eoutdurs Jo 
Jouo azenbs ueoul Jo uonewnsy (7661) “M'N'f ‘OWE! “f “‘AdSITA 


OE-1Z ‘UONMIIOSSP JDIIISIIDIG UDIIAaUPp ‘Spoyjay YoADasay 
Aaaing uo uoysag ayy fo ssuipaa0dq ‘S[OpOUl |[OAd]-N[NUI 
uIsn uoNeWNss ore [TeWS (€661) “dA ‘VANOW 1° “d ‘LTOH 


"ELPI-LOVI ‘16 ‘UouDl20ssp jDIIISIIDIS uDIIdaUp ayy fo JoUANOL 
“S[OPOUI poxTU JeoUT] [eoryoIeJary ur Surdwes sqqiy uo sioud 
Jodosdunt Jo yo0]J2 OY (9661) ‘D ‘VTTASSVO 1° “df ‘LYAAOH 


suoi6es sayjad 1nod seuusAow sap anbiyosesaiy euuelseAeq UO!JEWISY :ObY 18 NO’ 


TLY-SSh ‘ph ‘sousuIg paddy 
fo jousnor ‘surjdures sqqiy uryyiM Surdures stjodoneyy uonsalar 
oandepy (S661) ‘OM ‘NVL 3 “O'N “LSAd “UM ‘SXTIO 

“€6-SS ‘6 ‘a2UaIIg 10914811DJG *“(UOISSNOSIP d2Ak) Jesteidde 
uy ‘WUOHneUINsS vole [RUS (7661) “MN ‘OVA 1° “W ‘HSOHD 
‘Q9LI-LPLI ‘02 ‘spoysap pun K1oay | 
- SONSIDIS UT SUuOYWIIUNUWOD ‘suCTOedxa IOLIa\sod yeuIsIeW 
Joy Surdures sqqi (1661) ‘Wd 'V ‘HLIS 3° “a'v ‘GNVATaD 
"60P-86€ “SB ‘UOlDIIOSSP [vI1]SDIS UDIIWaULp 
ay fo JouAnor ‘sotyIsuop JeUIsIeU SuNe[Nofeo 0} soyovosdde 
peseq Suydueg (0661) ‘W'd'V ‘HLINS 3 “a'V ‘GNVATHD 


[eH pue ueudeys :uopuo7] 
“IOT-Spl ‘Cspg ‘seseyfaderdg “gq 19 ‘uospreyory “g ‘S¥TID “U') 
BINIDAT Ul OJADD aJUOPY UIDYD AoyADPY SUR “SpoyoW paseq 
-surjdures Sursn uorjeuruuajep jepow (S661) “AV ‘GNVATAD 


“UST TAD OSpuquied ‘Ae UOsUIgoY “UITBOH SIGN Jo 
aININSU] “WU sonsHEIsOIg DYW OE'0 UOISIAA ‘IndincC Suyjdung 
Sqqin sof asnmyfog sisdjoup indjn¢ pup sisouspiq, auassaauody 
‘VdOO (9661) “HM ‘SHNIA “WW ‘SHTIMOD “N ‘LSdd 

"QE-8T “EB ‘UOUDIDOSSy 
Jodsyojg uvridaup ay] fo Jousnor “eyep oai[aIes pue Addins 
sutsn vare doso AyUNOd Jo UOTdIpeid JO} [apouwl syusuOdWIOD JOLIE 
UY (8861) ‘VAM ‘MATIN 19 “WY “MALYVH “d'D “ASALLVA 


HIHdVaOOITala 
‘(SS-HS “P661 
‘gua T.) (0 ‘Q) Ins yntsod yso (LU), 19 QO <u onbsind 
nee 
> u ny =— = 
O> (UL) sua (lyy,6 


902 


sioye suoae snou ‘(W) J/( WU), J = (U)A WoSs 


: (U) J ue 
«(!y1=! ee SE YY ps ap 
(2 LD) 80] + (Y) 80] uw + (h) 1" = (Wye 
‘JUSUIO}SOJTULTA 
nor Ue 
O° (uyye 


onb Ja4jUu0u op 

juesyygns yso TJ [LY ‘vy ‘A ‘2 ‘g (4 |U] 80] = (UL) y WOg :eano1g 
‘U op onbrumyj eso] 

QaRdU0D UOTOUOJ UN so [Y ‘Wy ‘A‘2‘g‘{|U] :ouMaT 


‘TV 


1=¥ 
(yhZd+* A,X op), HOt X,Xop?) «4 ‘ 


JL 
2D 


1=4 
(yh ZmO + 4, Xo, (Od + XxX, Xe) Ss ee mi 


(qd + X, Xe ) bokZ@O aA 1X@)* 
! 1 1 . 1 1 a 1=1 5) 
(yd Zmod 2 4X), (Hd us XX) <r oF 


I=7 


1 1 a 2 
fae + XX) <G - 


s 1=7 3) 
| wy? MO ‘yh A I'D KS 
D 


2) ! Late 
ay? “0 ‘ah A107] CoE 
9) 


a 1 rete) 1 
(> oO ‘ok ADA KT = CA 
5) 


:red guuop isa ‘g op 
WOLI9}sod & DOURLIRA BI Op PSIT[aMAOLI[ G-oOvY InoyeuNsa J 


‘TV 
AXANNV 


“epeues 
NP d1U93 Ud 39 Sa][aINJeU SadUAIOS Ud SoYydIOYEI ap [Iasuo|D 
np uonusagns oun sed onsed usa saAndde 939 yuo xnearn 
SON “€ UOTDES FR] & YSI[N saguUOP op s[quIaSUd,| TUINOJ 
OAL P ([Ispiq) OllouRL OP OLY OP s[eIOpoy ousIOAIUL) | 
op BINO|Y “4 Inassojoid 9] JOIOJOWIOI B JUSUETeSa suOUa} 
SNON “Son suoNsessns jo sonbsewiss sInay op Inajoep 
-91 9] 10 SINDJCUTULXS XNOP IOIOIOUIOI BE SUOUD} SNON 


SINAN NaN 


(6661) ORY 19 NOX 
DOAB 9IULIOJUOS Ud ‘a}gnbue.p ued ap spiod say J3I139)UI 


S0¢ 


A.p ulye juaatnsinod as xneael sayz ‘onburtdus sieiq 
sues dIIegUT] INd}oIpeid Ino|[owl Np spoyyeul ey Jo onburdura 
uudIssheqg spoyjeu ve] o[duroxa Jed ‘ayeuolsal UOTRUINSS | 
Inod sagsi[Nn JUSWIEZIe] Sopoyjoul sone .p eB onbryoressr1y 
ouuslsgheq epoyjeuwl ve] Joreduioo op jueyodum juoWsyes9 
so [] ‘onbrysseigiy ouusrsgheq asAyeue., op jueyoduit 
yoodse un jso dJQpoW np x1oyd YF] ‘*xneuoIsel sonowesed 
op UONRUIT)sa | Inod suoneniis ap x1oyd uN yno} ev onbrydde ;s 
gjeigues onbiyosessiy ouuatsaheq spoyjeu P| ‘SUIOWIUPaNY 
“SqQID op sseUUOT[HULYDE Pp BpoyJeU! eI B SddI] odeUUO] 
-[nueyog pp suonsanb sourezss Jo a[Qpouwl op sazquresed 
SQ] Ins ond ev sUONQLSIp ap x1oyo 9] afduwioxe Jed ‘suo 
-PIWUI] Sop yuoWeTe39 syOdWIOD onbiyosessty ouUsIseheq 
Opoyjoul eT ‘suONeNyIs soayne.p e Jajqid os sed ou 
jUOLINOd suorjenys saurteyiao suep soudoidde sajapow soq 
‘JRUIJ JeI[NSOI J] INs JoJJo UN yUOINE salrel[Ixne sUOeULIOJUI 
SOPI[OS ap douasaId PR] 9 SUOIZAI Sap SITQUaSOWOY |] 39 S[[Ie} 
RY] SMUT]! JUDWIOT[ONYTA 911xa]duI09 oun ,p sudIsaXeq soTep 
-OU! Op apie | B SoguUOp szJ Jo}snie .p juasWouLIEd snou sqqiyn 
Op WseUUO][NHULYDE | SUTWIOD (QUUDIAOYILU SUTeYD B O]IeD 
UO) WOOW 2d} ap souuatsgheq sopou}eul Sap Is SUIQUI 
‘sJURSIVJSHeS S}e}[NS9I Sop 11U9}GO B US} Op SUOIZaI So] 
$9jno} Inod gudoidde sfapow un WWINoJ ap e]1oej sed yuo 
-O1I[9 1S9,U [J] “XNROAIU SINaIsnid e salapow! ap opre | & ‘ayeu 
-O1391 UOTBUIT}SA | Inod sanbryoreseiy souusiseAeq sopou} 
-9UI Sop gjuasgid suoae snou ‘gsodxe juasgid 3] sueq 


NOISNTIONOOD ‘PF 


09.9€ 92°09 Or'S9 §=6—68S9)—LBSO—s BSD Ol 
C8re 98S 6679 8E'€9 9PE9 9P'E9 6 
1998 O9LPI OI6SI O66SIT O0O9I OT‘O9T 8 
BSL¢ S8r6 §6Or Ol O7ZOI O€'ZOIl O€‘ZOI ik, 
06°7S IZL8 96°76 €8'€6 €0'v6 €0'V6 9 
CL'99.  OL’EII «=(OFIZI §=«(OSIZI «=(OLIZI «609 IZI ¢ 
78°87 I@8b  e9'IS p07 80°7S 80°7S v 
09°0S €L'98 91°66 C@rP6 6HP6 8 8h'h6 e 
6061 PL le LOCE LIVE SIve 6I'E G 
6777 =O PILE = H9E coor Ol0r 600% I 
Ol I é ro uae 100°0  1000°0 noo 
q='v‘('q"D)O1 


aguiljso oseuUOT[HULYIE ,_p 
INoL9 | <p ssoueLIeA ap uosteredui0D 


g nesqey 


"SOANPULIOJUI UOU LOLId & suONNgLNsIp sop inod 
sonua}qo 1uOs Inb sa][90 ap sajusJayJIp JUoWayqetogidde yuos 
'O ap WoLa\sod e suoneumsa sot ‘Q[ ='¢ = ‘vp a[duioxa 
Jed “g 19 'p ins woud & suoNeULIOJUI soprjos ap souasaid 
ua ‘stojainoy, ‘sained snd yuouuetacp _'9 suoneumnse 
sg] ‘uajuowZne ‘g 10 ‘pn onb ainsow Y ‘suONeUINSS soy 
anus souarayyjIp op sed yuowonbnerd astxa,u [I ‘(100 =>) 
sained yuos ‘g 30 ‘pv onbsio, ‘9 neajqei of onbipur. 
AUNUOD */o Ins LoLd B suoNGINSIP saUaIDJJIP sap opre,| e 
79 ap UolIaisod & suoTeUINse sa] auasgud g Nea{qe) a] 


0002 e:quiacgp ‘ajeanbua,p senbiuyos | 


€19'0 Srl'0 OSL‘0 09L'0 09L‘0 09Z'‘0 Ol 
0640 S60 IIOT viol FIO SIOT 6 
LEe'l Se oy SEED SIs CIP Paesir' tl 8 
6LL0 160 O€60 I€60 £60 £660 L 
6690 9780 OEs'0 §=—OL8O~—Ss(«OL8‘0~——sCSTL8‘0 9 
CLE Ie aeehe cor Cl sesy cc. FSSP'T  S SSr'Z S 
COO LOC TEC SIT T SY 077 TF" 077 T v 
LUGO’ SST Srl LON] LOPE -LOLl € 
COV Or =VE90r FLILO “CLO VEL 0 * *vZL'0 6 
66r0 £590 999'0 8s9'0 = 8590 ~—s 8590 I 
Ol if ' oy a 1000 —1000°0 uoioy 
g= D(C G*D)OI 
S9UIso sseUUO][NUeYSE ._p 
Ino9 | 9p UOLa\sod e sooueLeA op uosteleduiog 
g neayqey 
a) 10 


'D OP SUOSIEUIQUIOD SaJUDIDIJIP We1oUOduI09 a1puojoidde 
snjd epme euq “g='p seo 9] JUowWenbrun sigpIsuOd 
SUOAR SNOU ‘apni9 aOU SURG ‘AaSeUUOT[NULYSE ,p INaLI9,.| op 
SOOURLIRA SJ Ins WOLId & s}UsUIOUSTIOSUaI op snjd sUOAR SnoU 
IS AD 8] BUJZNUOD Inb 29 Ud sUOISaI Say INod suoneUINss sop 
$}e][NS9I Soy] JoJOI[gWIe suOANoOd snou onb anbiput efa_ ‘(Ad) 
UONIIA op sess sjyed snid op e yuoueUI 39 ‘soAeUL 
-1oJul snjd yuouUdTASp ,9 Ins old & suolNqLsIp say aonb 
dINsSoU & JUSNUTWUIP uotiaisod B SOOURLIEA $9] ‘¢ negiqe) 3] 
QIUOUI 9] SUTWOD ‘stojainoy, “g 19 ‘v ap xIoYo ne sa{qisues 
sed UOS dU Sala ‘satquis Sai} JUOS soTeUOIZaI souUdAOUI 
Op suOT}eUT}sS soy onb JuoWaIIe[D onbipul p nes[qe} oT] 


76'6 78'6 6L'6 08°6 6L'6 6L'6 Ol 


CCiiecoi MeCOUL FSEIL PEL ~: vEIT 6 
OS OLeLe Ol, SCO! JOL01 ~ 0c Ol, 67 01 8 
LS‘6 €9°6 85°6 65°6 8S°6 8S°6 L 
ScOlgscOlm IcOly ECOL Icot ZO! 9 
LESese LET OL SS L1 | CES LY ~) 9821 S 
I90I 601 S60I S60I Sé60I SéO0I v 
SUC OcE TNO EL IOS | OO'SL 00ST o 
79°6 78'6 €8°6 18'6 >8°6 18'6 C 
LFOVIRISG OLD -PZOLY MESO. ECOL ECOT I 
Ol I ‘0 10°0 100‘0 ~=—- 1000 ener 
'g='0‘('q“0) 91 
SagUITIS9 sayeUOIZ9I souUdAOUI op UOSTeIedUIO_, 
p neque 


“sajuepuodsal09 LIOLIa}sod & SooURLIvA So] a1]UOUI 
¢ neg[qr} 3] 32 ‘ 70 Ins WoLd & SUOTNGLSIP SaJUsJIJIP Sop 
aple | & Z B[Qpoul np sipeo aI suep WoLIa}sod e souusAoul 
sap ajuosoid p nesjqe} oT ‘soaeuosul woLd e suornq 
-LNSIP Sap SUIWIOD sagigpIsuod ang juoAned (Q[ ‘OL)OI 
12 (I ‘[)D] enb spur) soaneuosul uou Loud e suonnqiy 
-SIP Sap SUIWIOD saaJepIsuod ang juoAned ([0‘0'10‘O) DI 
19 (100‘0 ‘100°0)DI “(1000°0 ‘1000‘0) DI Isury “saiied quos 


'¢ 12 'p onbsioy 'g 10 'v woLd & uoNeULOsUI | JUaUTWOp 
Z/CP'X 14), CPX - 14) 12 “Wu uorueyos,p syayyo $9] 


(eine IX RU) 


CEX - n= +'q uns D)D ~ [oA K|'2} 
onbsing 
‘OL 39 I ‘10 ‘T0‘0 ‘100‘0 ‘1000‘0 a11p-e-189,9 ‘g ='v ap 
SOJUSIPJJIP SINO[VA XIS 9XIJ SUOAR SNOU ‘7 aJapou! Np sped 
9] suep ‘¢ 10 'D ap xioyo ne LOLa}sod & suoneuse sop 
SIU[IGISUdsS PB] IOIJLIOA Op UITY ° ;9 op sondea soourssreuuo0d 
SOU Ja}g[Jol & UOSeJ Op sdURLIVA op sajyuvsoduos say Inod 
sonbinguiesed sing[eA So[qiey San op doe saguidosdde woud 
B SUONNQLISIP Sap 9SI[IN SUOAR sNoU ‘IsUTY *[Q0‘O B SeTe3E 
SaIstoyd 919 JUO ‘g Jo ‘v ‘saguUOp sap ascyeUR aNOU sue 
; 79 sseUUOT[NULYSE Pp SooULLIVA soy Ins WoLd e& suonng 
-I)SIp Sop xIoyd ne Lole}sod & sauudKoul sap siI]Iqisuas 
P| JONeAg SUOLIOWIIe snou ‘Jorjnomed ua ‘ 70 so] Ins uoud & 
SUOTINGLISIp sap xIoyo np sajqissod sayy xne suosselgjul 
SNOU SNON “aJUsWIsNe ‘oO saoULLIeA sap sajuesodui0s op 
SIQUIOU JI ‘aJUSWISNe Ww SUOIZEI op aIquioU aT anb ainsoul B 
‘snjd oq ‘aseuUO][HURYDE ,.p S2oueLIeA Sop ya[ns ne sajoexe 
SEO SOP I1U9}QO ,p 9]1O1JJIp sinofno} so Tt ‘onbyeid 
RB] sueq ° ;0 op woud e SeOUBSSIEUUOD SOU JOI9[JOI uodsey 
op sasioy> sonuuod sinejea sap quos ‘g 39 ‘pv no ‘(' q° D)OI 
@SIOAUI eUTUeS UOINGINsIp eT saae 'o no (‘g“n)D 
Wold & SUOTINGLSIP sop d0av sajyuepuadepul sagsoddns 
uOs 72 = '1 INdUI,p SdOULLIVA Soy ‘Z B[QpOU! 9s] suLq 


gu[IqIsUas ap ashjeuy p°¢ 


€Iso 6980 Or 
750 «L660 6 
Zr8'0 OCT 8 
o8r'0 = 860 i 
I8h0 =: O6'0 9 
8L8'0 «8ST S 
+090 101‘ p 
8rS'0 «060 € 
g6r'0 = Z98°0 Z 
90S'°0 —s-LI80 I 
awl! Glew 
uonejnuis op sedA}-sinouq 
¢ neq 


‘7 nea[qei ne ou aguinse odAj-inea | e onbyuspr onbsoid 
4sa ¢ Neaqu} ne : if ap uonenuts op odA}-IneLe | ‘Isury 
‘tIola\sod & adA}-InaL9 | Op INVUII}so UN JUSUaTe9 Se 

‘1 ap uonernunts op odAj-Inaua | onb JsUUONUUT 9p JUSIAUOD 
ie ‘g]JouUONenduIOD ouUaIssheq oscTeUR OUN SUBP LOLA} 
-sod ve sauuaXour soy Inod poliety Oued SUONPUINSO Sap IU} 
-qo Inod asiqn J uo <absi0] , an anb ajqeis snjd iso ec ya 
‘Isuly “UOoneRSIT[OMye[ g- ory “ Op aseURAR | oULNUOD 
mb a9 ra ee 9dA}-INdLI9 | BP IOMAUD %C/L & %WOS ep 
alussgidol ( ya ap odAj-Inala | ‘seo so] sno} sueq “suoIs 
-91 Sd] $ojno} nod , at anbuidwe inayeunsa,| aonb s[qrey 


suoi6ai sayijed unod seuuehow sep enbiyoeseiy euualseheg UONEWISA -O8Y 39 NOA 70Z 


snid dnooneaq uonrnuis ap odAj-1ne1e oun ayodwios in 
PSI][OMYov[g-ory Inojewinsa,| aonb yuouromels onbrpur ¢ 
neg[qe} VJ ‘uoneNuIs ap sodAj-sinau9 | aNUOW ¢ NeI{q 
21 2T “{[y2 “WO ‘oy AIST X} 1 (dX) OP voTN 
-uevyo9 ,p sodA}-sinaa so] juautsanoedsar yuos tnb ‘ awe ‘1 4a 
@e 1 ap uonejnunts op sodA}-sinad so] 9[Nd[Vd SUOAR sNouU 
‘UONSITJOMYAOL[G-OvY L] IP S}ojjo so] JOIpNe.p UlsYy 


180 6L6 O€ Il I00I Ol 
10T “PETE POLS © S101 6 
6I'T O€‘Ol 661 I8‘ZI 8 
L460 = 856 ~— PTL CZL'B L 
COOP ICO ese lerC rt 9 
isl. “PEs iV” Serr “7S 61 S 
{iT Ser M108". €69 v 
80 1O'ET SI'rl 8P'el € 
ss0 «86S 789s TOL Z 
180 2 COL £56. SSO LT I 
Fe Ot eee ta uoIsay 

SoTBUOIZ91 SoUUDAOU Op suOHeUIS| 

Z nvIGeyL 


‘sayoorp sodA}-sinole 
saj onb sayned snjd dnooneagq yuos LOLIa\sod e sadAj-sinalie 
SY] ‘Sa}DOIIP suoyeUIse soy aonb saovoryya snd yuoWs]q 
-elogidde jUOos saasi[apowl suoneutse soy onb yuoWIoIIe[O 
auoW Z nesqe) J] ‘SuOIZaI QI Say Inod sajyuepuodsas 
-109 $a}D1IP SadA}-sINdaLd sa] 39 (UOT[HUBYSE ,_p sSouUdXOW) 
SOJODIIP SUOTJLUIT}SO SO] Q[NITeO SUOAR SnouU ‘UOsTeIedWIOD P| 
Inog ° aan il op opre | @ sanuajqo suoNeUltss soy JUoWoNbrun 
ajeusis DUOP SUOAR SNouU ja ‘saTjanjouod suOTFeUIT}SS SOUUQUI 
sl onbsaid WWassTLINO| , ya OSTT[OM YOR] G-OvY INdJewNso | 

Pint 1 onbriiidura inayeuinse | anb snuoul & apnid oNON 
Be eae ae Uotse}sod e sadA}-sInaLa sa] 39 SO9UIT]SS 
Uoa}sod & soyeuorsei souUudXOUI Sa] oI]UOUI Z NeIIqQR} FY] 
‘JUSWZTNEs Z S[QPOW NP UOTOUOJ Ud sayeUOIZaI souUdAOU 
sop suOleUINsS soy jJUReUdJUTeUT sUOJUassId snoN 


UOHCUITASI | 9p FBINSIY CC 


OdD Uosteseduloo op goe]) :ajgpoul Np uONIeTVS *] ens 


€ pue ‘2 ‘| sequinu hq peyeripuy siepowy 
ogz 002 OSL 001 os 0 


200 


SABRE 
a a 
nN 
ye 
~ N 
a SS Sere 
we 
pea 
“ ne 
7m Be 
Ve ie 
be aes 
pete ON 
700 
Odd 


€0¢ 


“soxouuod 
Wollajsod e sadAj-sinaLid sap Jo sayeuoIs91 souUdAOUI op 
SI9SI[PPOW SUOTILUIT}SA Sap ITU9}qGO Inod Z ajEpoul 2] ssTjyN 
SUOAR SNoOU “UanbgsuOd Jeg “seguUOp sou Inod jusUNA}snIP _p 
Q[QPOUI INIT[IOUI a] 3S Z BJQepour a] onb suonj[ou0d snou 
‘Odd 9081) J] 10 [ NeaTqQuI 9] SUeP 9ANO]) UO,| anb aseuUO] 
-[QULYD9 Pp DOURLICA P] OP SUOTJBUIT}SO Sop nud} a}dWIOD 
"saguUOp ap afquiasua oy0u Jed 
so9}00dsai ustq sed ang-jnad yUOs ou sa[Qpoul sou ap sasey) 
-oddy sa] aonb onbriput nb 90 ‘sayepoul sto so] nod OgD 
sIng[ea soyijed op Issne e A [J ‘Odd Sinafea sap so Inb 39 
inod | a,apour a] anb Inay[Iour yUSUIIISY] ISO ¢ B[QPOU JT 
"€ 19 | SoJQpou sap sa]jao aonb epuvis snjd yuowa;qeiogidde 
4So Z BJopowl aj Inod OgD sineyea op gyWofew oun .nbstnd 
‘INI[[IOUL V] JUOUIOIIV[O ISO Z B[QpOW FYI *[ sINsIy ey] 
B SI[QpOu SION say Inod OD 9de) uN suojuasgid snoN] 
(S661) pursleD Sine eg|[teigp snjd wots 
-SNOSIP OUN BISANOT UO “(8T) Ud |, ,9 Jed agoeviduias Eso 
IAB Odd SI B[NIed UO ‘¢ 9 Z Sa[QpoOW Sa] Inog ‘(¢) Jed 
SQUUOP I[VULIOU 9}ISUSP Op UODUOJ eI 3S9 (Jo “| "4) f no 


sqo ‘fi 
(2? ‘gl Ae 1 12. 


I DSeih 
(81) i = "oad 
_ 


“ins wD sag[no[ed IuOs OD siNeTeA 
$9] 19 iC log |fa) f= (i fe “d ( 7 \"4) £ ouop ‘soyuepuedgput 
JUdUIa]|SUUONIPUOD yuos {4 sa] onb jueuajUTeU SUO}OU sNON 


ong er "ay f 
oP dP(A|29 “SF fa) 
I 
ial 18 (79 “d ‘ays 
PP IP(A|79 “PSF wes! 
I 
fi) 
ea OA: 


SUOAB SNOU ‘| s[apow np 
QIPed 9] SUep ‘IsUTY ‘so[duUIIs saguUOp se] soyno} Inod O_D 
So] Joynoyeo suoAnod snou ‘sqqid ap InouUOT[HUeYDE,| Op 
SILOS P| Op Opie | YW ‘S9gAJOsqo saguuOp sz] Inod yuowaysnfle 
INI[[IOU UN JUasstUINO; sagazia sn{d OgD sap iueOdUIOD 
SO[QPOU S92] ‘SOgAJOSQO SadUR[QUIASIeIA Sa] anb yUOs dU Odd 
saj anbsing ‘agAsasqo [duis sguuop ey susisop *“* bi no 

Ce (A) Vile Lveye i "Odd 

SUOAR SNON “S2[QPOU! SIO. sap uNdeYd Inod (QgD) aT[auuon 
-Ipuod satjoIpoid @QUUOPIO IURSIP-10s P| ‘agalasqo aid 
-UWIIS aguUOp ek] Inod x | "4) f Op INs[eAa ek] SUTUTeXd SUOAR 


0002 eiqwesap ‘ajanbua,p sanbiuyoe | 


snon “A gnes saguuop sgI sano} ausisap z no «(Mz |MOs 
Q][oUUONIpUuod gyIsUap eT Iso “ Inod sgstoID UONepTTeA 
op ausuap eT “4 afduits eguuop onbeyp snod agsto19 uonep 
-I[WA ap dATJoIpgid gyIsuap eT g[No[eo suoAk snou ‘aTapoul 
anbeyd juorndde saguuop sa] JUSUIUOD JoUTUeX9 .p UITY 
‘Teumndo juowsjsnfe .p sfapowl uN ,p uONd9eT 
-9S8 BP] OP ONA Ud SgSIOIO UONepI[eA Op opmyg oun JueUaUTeUI 
suojuosgid snou “juoWosstpuojoidde .p asin3 uq ‘saguuop 
op s[quiesus oNou Inod jeundo s[gpoul af ong WeLNod 
‘Q[eS9UI INDLIO,[ 9p SULLIVA L] IP B[QPOUI 9] SJIP-e-1s9 9 
‘Z 9[gpour aj onb juonbipur |{ nesfqe) np syeynsal say] 
sopor[dgp quos A 39 98‘Q/ = lane’ © & sgfese ong Be JUapUa) 
7O Sa] $9INo} ‘¢ s[QpOU NP sIped 9] sueq “(Y‘U)DO 
SUMWIOD SBgNGLSIP sautoyeaTe sofqeuea sap juos 'o sa] onb 
asoddns ¢ aJapoul ax] °g 19 ¢ SUOIZEI s2P ulas ne eee 
sayoj snd op yuenbiput ‘p6‘6ST = (ene 19 SOIZI = ae 70 
4so a[qenbreuwias snjd 9] 3e1[Nse7 71 ‘OOW suoneumnss sop 
soULpUd} BE] SAAR JUSPIODUOD (9 '9 So] SUOISST sop yoodse | 
SINSOUI SUTeIIE 9UN sULp JUSNUOU! (0 'O aguIlse Ina | 
Op soouRLRA SQ] ‘7 B[QPOUI Np aiped 9] sue ‘anne | e UOIsaI 
oun.p /o oyeseul Inala | ap soueLeA oun asoddns Z afapour 
Y] ‘SUOIZaI soutey9. Inod OOW suoneunse soy aonb ajqrey 
snjd dnooneaq inayea ‘98" 9L =, *0 Jed *o aUInsa UO J9 
‘SUOIZQI Sot Soyno} Inod ? 70 gje89 InaLo,] ap souerea oun 
asoddns | a[apoul a] -suoISal OL Soy tuued suoneLea soyoj 
op uonbipul QOOW suoneunse soy ‘] neojqe) a] sueq 


COAL ZL‘S9 98°9L 98°79 Ol 
LS‘OL LE‘e9 98°9L ‘6S 6 
96°L6 v66Sl 98°9L S9'SET 8 
v7 LOTOL = =98°9L L9‘101 L 
8L'6L Se'v6 98°9L vL‘06 y 
OL‘L8 SOIZI 98°9L 86°STI S 
LUL9 10°%¢ 98°9L EL'8y v 
8S°6L LL'¥6 98°9L 9718 t 
€1'%9 ve'VE 98°9L CLG C 
9°¢9 81'0r 98°9L LI'8€ I 
€ARPOW 7 ARPOW 1 eIRPON OOWN uoIsgy 


SBCUUOT[NULYIE Pp IND ,| 9p SOANBUINSS SooURLTe A. 
[ neaqeL, 


‘OOW suoneulnse 
so] onb ouUIQUI op SaTepoUL SION Sap sped o] SUP oseUUOT[N 
-ULY99 ,P SOOULLIVA Sap Sa9STT[OMYoR[ g-OeY SUONLUITSO Soy 
enbipul | neafqe) a] “juauta[Nes UOIsaI Be] e Sordoid saguuOp 
Sop opie | & UOIZ9I1 onbeyo ap ules ne sseUUOT[NULYI9 ,_p 
SOURLIVA SOP (QO) SOIIBUIPIO SeLIeD SAIPUIOU Sop suON 
-BUII}SO So] g[NdTed JUSUaTeS9 SUOAL SNON ‘SO[QPOU! STO) 
SOP sIPed J SULP BSeUUOT[HULYDE Pp SeOULLTRA Sep WOLIS\sod 
B SUOILUINSA Sa_ NUDIGO PJOge Pp SUOAR SNoOU ‘ssgUUOP Se] 
aisnfe sjapour Jenb Jouruajep op ulpy ‘oseuUOT[NUeYO9 .p 
soourueA xne juenb sajuaiajjip sosgujodAy,p uworouoj 
Ud Sa[Qpoul ston gsodod suoae snou ‘7 UOTDES BF] Y 


ajapoul np uoNI7]9§_Z"¢ 


‘ISIOYD B[QPOUL 9 Ins sagpuoj soTeuoIse1 souUdXOUI 
Sg] Inod sagsi[gpoul suoNeUITse so] JayUsgid suOTTe snou 
yo ‘soguuop ap sfquiasus | Inod [ewido s[apour un IIstoyo 
pioge Pp suOT[e snou ‘alIe} 90 Inog ‘jUoUoANDedsar “x 49 
'y sourerjrxne eedueue se UOIS9I OUIZI-? BT an uonejndod 2p 
souuekou say juos “x 39 "ly no “eg '%y ey, Ng!ly + g='g x 
=' afeuol8or ouuakow vy] JoWINsa e suOYyoIaYo snoN. 
‘Q[QPOW Np UOneUWINsSa | op Jo asATeUR,| 
OP SNA UD SA9AIOSUOD Jd S99INI9Xd 939 JUO SUOTIPIOII SayNE 
000 ¢ sind ‘suoneisi 000 Z op e3epol op aup spoued oun, p 
SINOD NV 9INd9X9 919 PJOgE Pp B SQQIH op InouUOT[HUeYDE | 
‘g[gpoul onbeys Inog ‘owwess01d 9] suep sayoep ane 
JUSATOP SaNguIvIed sop soTeNIUT sINaTeA Soy Jo Loud ev suo 
“NQUISIP SIT “SON esesury np opre | & siLOgp JUatos salap 
-oul sou onb nainod sqqin op eseuuoynueyog | Joynd9xa ,_p 
2 Sollessso9U sojQ[dWIOD sajjauUONIpuod suOTNGLNsIp 
S93] SIINIsUOD op JouLIad SON g owUWeIZ0I1d a] ‘s0UdBIOA 
-U09 BI Op UONeNTeAY,| Inod (966 SOUTA 19 SOTMOD ‘3Seq) 
snidg VAOD Uonsuo} eT op 19 (966T “1109 10 JyeYTaBards) 
SON swUeIso1d np opie] e& sajepoul siom saz Iinod 
SQQID Op INoUUOT[HULYDE | DIANA Ud STW SUOAR SNON 
"OLI9\sod & UOTRUIT}SO | INS [eUITUTUL 
JOJJO UN IsUTe JUBDIOXO Saja nb 39 souOsIUN WoLId eB suoNNq 
-L)SIP Sop UdIQ JUDUTATQeuUOSIeI JUOYoOIddel os sonQuiel 
-ediodAy so] Ins sogudoidde woud e suonnqmsip sonsea 
soy onb 99 & suOpuUd}e snou sNON ‘a][eAJo}UI puess uN Ins 
SULIOJIUN UOLINGINSIP &] SIIP-B-3S9,9 “(QOOOT‘O)N suUIOD 
juswUepusdgpul senquisip yuos _Y 30 lL onb asoddns 
uo 39 “(Y ‘UW)D ~ '2 ‘¢ aJapour a] nog *'2 soy Jo “a Inod 
agsoddns 3s9 (100‘0 ‘100‘0)D Pues WoLd & uONNGLysIp 
oun ‘7 30 [| So[QpoUl soz INOg “sdoURLIRAOD op sdLNeUI 
R] Op InopuLss op oIpsoO | Inod asreunuyeid UOHeWIse onOU 
ajuasgidar tnb 99 ‘[Q0‘0 & xnes9 xneuOoseIp UOU s}UDUIIT 
sop 19 [ & xnes9 xneUOseIp s]UOWIZ[9 Sop dOAP a}LIDOp 
SO Y OIIe[eOS SOLNLUI VT “(9661 SHIED 10 Isog ‘seWOYyL 
‘Jayfeyjosords) 7 op suel a] ‘¢ = 0 ‘afqissod onb aynod 
Issne OS UONNGLNsip 309 onb Inod » 9yOqI] op soIsep 
Sa] ISIOYD SUOAR snoU ‘OLId & soURSsTeUUOD onseA oun 
Jojuasgidal Inog *;_@ = G ISIOAUL! SOOULLILAOD Op SOLU P| 
inod agijtoeds 3s9 ‘Cy ‘p)" 4 UeYstAA op UoLd e uoNnqmsIp 
aun ‘(007 ‘00Z-)Q eduiexe sed ‘oguuop ong Weunod 
QBIE[ JUSUS[QUUSAUOD JI[BAIJOJUI UN Ins sUOJIUN LoLId eB 
UONNGLsIp oun ‘gyITIQissod anne SUIWIOD ‘SOUL[GUIDSTLIA BT 
sed ogAndde uorsai ey] Inod yuawayeso] ouLosrun yso Loud & 
UONNQLISIP LT 39 ‘UOIIAUD CQOTF & pUddsaLIOD % C6 op LoLId 
B o][eAJoquI un nb ayos ap ‘QQ] odAj-1Ie99,p 19 Q suUSAOUI 
gp sojuepusdgpur sayeuloU so[qeLea sop Jong sogsoddns 
yuos 14 O24 «114 OTL 2X auop ‘(01 “01 01 4,01 ‘,01) 8eIp 
=gq no ‘(q‘o)SN~A AMS sUTUIOD sagrIo9ds 
quos snuuosur soneguiered sop ins sogudoidde woud ve 
SUOTINGLSIP sansea aq ‘seguuOp sep osAyeue aNOU Inod Z 
U0N]9AS PT B SPINOSIP SO[QPOUL SION} So] 9ST[EN SUOAR SNON 
‘g]eqO|3 UOT[HURYS9 Pp OUUSAOUL vs ap INojNe 
genus juoWaTeS9 1So 'z op INo[eA eT] ‘UOIsg1 onbeyo suep 
asevugui Jed soInjIOA op USAOU SIQUIOU 9] SUILUOD OIUTJOpP 
uolZal e] op Neaatu ne s[qeutea oun ysa 'Z 19 ‘( ‘0) *N ~ ‘A 


suoibes seyjed unod seuuehow sep enbiyoesgiy euuaiseAeg UOEWISA -O8Y Je NOA ZO0Z 


SUIWIOD 9NqUISIP uorseI awgt-s Bf Op sauoIg|e $19JJ9.P 
imayoen of so, (7a Ta‘) = “a ‘saxty WOIsser89 op sangui 
-ered ap nuuosut najoa af ysa_, ("A OA IT, OIL 1) = A n0 


co ane Moldy 0% _ Rd ala gy folly Ones lg 104 4 Ox — 10, g 


QUIWUOD JOSI[QPOUI ISSIR] 9S 39 UOIZEI 
UIRI-1 B]  JULpUOdsaLIOD aMOIeg{e UOISsaIZ9I Op JOTI 
-J009 9 189 ‘g “(L[) aBeUUOT[NULYDS,p a[apour a] sue 
‘Z UONDAS BI B SAQINOSIP INDIO ,.P OURLIVA PI Op So[QpOu 
SIO soy Jed agstogid yue}9 UOTNQLYSIp es ‘aseuUOT[HUeYDE ,_p 
INdL9,| op s[quuea eI 3S9 “a 19 saANOedsaI [eqo[s UOT 
-ULYD9 ,P SOUUDSAOUL SINB] 9p INoyne ssenuso quos “x ap 19 
“ly ap sanajea Sa] “oseugu np Joyo 9] Jed jurone juepuodsar 
-I09 9ILILJOOS Op NedAIU d| DUSISOP My yg I UOIZ9I Be] Op 
aseUugU! sUIZI-/a] SUP sadeId op sIqUIOU Z| OUSISEp My no 


(LI) Ms Hn vq Mey 4 AG: ve ae = we AX Le Lag 


Jed guuop 
jso OSvUUOT[NULYOE .p sJepoul a] ‘aseusUI Np Joyo 9] Jed 
JUIN}e 9}LIL[ONS Op NVIAIU J] SueiTe zB 39 oseUsW UN suep 
saogid ap siquiou 9] ausisep 'x no “x 39 'x omp-e-1sa,9 
‘9U1UN | Op NRSAIU Ne SoIIel[Ixne sopqeLeA xnap & A [I ‘uoIsel 
OUIZI-1 BT SURP aseUgWI sWIQI-/ Np nudsadl ZI ‘ he 0S 

‘uoIse1 onbeyo suep adults alojegTe oseuUOT[NULYDS 
Jed sonuajqo oseuuo][yueYyse.p soyun gz Jueyodu0S 
SUOIZAI CQ) OP B[QUIASUS-sNOs UN JUSTUOD SaguUOPp ap ds] qUIAS 
-Ud SON “UOHeNSNIII,p oy ojduis e ‘esAyeue o0U sUep 
SOQUUOP op [eUIZLIO sjquiesUd,| Op o1ed s]qQrej ouN jUOUT 
-onbiun gsijgn suoae snou ‘juenbgsuoo seg ‘sodA}-sine1e 
sop uonuajgo,, Inod ‘onbuidwe gs TqW np epoyisu 
RI & JUDWSITEIUOD ‘puvIS OS SUOIZ9I Op sIquioU a] onb sed 
oBIx9 .U sokeg op onbiyosesery spoyjou eT “a[duuts ar1o0jyegTe 
eseuUo][nueyse Jed uo1ga1 anbeyod ap saain juey9 aseUuUOT[N 
-ULYD9,P SaIUN Sa ‘SUOIZ91 Op] JUSWOdUWIOD BINOY|] 19 ITOH 
OP So[BUISLIO saguUOp Sz] ‘[ISsig Np (SUOIZ9I) S9]UIOD Op 
. SOSCUSUI SOP NUDADI NP UOTJCUIT}SA | QI9PISUOD SUOAR SNOU 
(6661) NOH 19 eANO| 19 (€661) BINOJ 39 OH JURAIng 
S2]QPpoul Sap 79 SaQUUOp Sep UOndLIISIG ['¢ 


SHAUNNOd Sd ASATVNV ‘€ 


[VU ‘A ‘2 ‘g ‘4|U] affouuonrp 
-U0d UONNGMISIp e] ap sed e suO][WUeYdE sop o1Inpold 
ep Uye sqqin sp InsuUOT[MUeYSY,| SUBP (C661) UBL 10 
1sog ‘SYD op gidepe jofol e aseUuUOT[NULYDE ,p spoyJoU! P| 
Jasin .p sqissod jsa [I “(ZY oxouur | J0A) L. op onbrwyy 
-830] BALDUOD UOT}DUOJ oUN 3s9 [¥ ‘Vy ‘A ‘2 ‘G ‘4 |L] onbsind 
‘slOJonNo], JUOWTNes soLeOT|dH [Nu ajUR}sUOD oUN eB Nbsnf 
nuuos 3s9 [Y ‘zy ‘A ‘2 ‘“g ‘4 |W] “E a[apouw! np alped a] sueq 


! C tat 1 
(91) = Ue + gl (gd - 1) x 


tal t C ut 2) 
Dae < AT + wv] cer 


LOe 


Jed 9uuop 
4so ;9 ap Wouajsod e suUdKOUW ¥] OP YSTT[OMYe[ G-ovy 
INJCUINSS,| ‘€ BQpOW Np sIped oI sued ‘7 s[Qgpow Z| 
Inod saguuop 939 juUO Inb satjao aonb soulioy SouIgUI Se] 1U0 
‘ri 10 ‘g op WoLa)sod e sinayeutsa soy ‘¢ a[apour 9] INog 


itech + uu) 6 ~ (WUhagily) Ga) 


CPD uw L(U) tly UA 2g a|ul (A) 


1 if 1 1 =f C 
[az -'“g)(Az- RU ies +y‘u+0 | 
mM ~ Pes ogre aD 
(((:- q+ 20) Zw) (d 6,243) 


(i-7 + '26,/2'03))'n ~ Feudal) aD 


era 1 tal 1 C xs 
Y,+ Cd X- 1 CdX — AT iat u 
[vy ‘u a ‘dx |'2] 
‘mM *T = 1 INdd (11) 
(6 + 'x,'"'2) Aiz6 + '4,'x'2) 
HAUS OLN Iya) INb zen Ey Wey arg] 
‘uM hares al = 1 Inog (1) 


o pin 


pul 


‘red soguuop 
JUOS € B[Q9POW Np sped 9] suBp sqqIyD op sseuUOT[HUeYDE 
un nod sajgjdui0d saj[auUoNIpuod suOoNNqUsIp so] 
“SONUUODUT JOS saT[o ,NbsJO] aseUUOT[HURYDE .P 
SQOULLIVA $9] JOSITSPOW op o[IOvJ sed jUSWII]eIOUNS JSd,U 
[I “2 anod uonetndod op sjgpow suruios (y‘U)5 you 
-g[ duis suosiynn snou ‘stojayno} ‘¢ uONDes e] op saguUOp 
sop askjeue | sueq “2 & Sol] SollelfIxne syuoWoUsIesual 
Sop JoBZIxo ned ejaDg “2 ap suIYWIEZO] 9] Inod uorIssaI3 
-91 op s[apour un juestogid uo “g ap seo a] suep sWIUIOD 
‘astyeai snjd uodey op ‘2 Jostgpour suoanod snou ‘9yI]1q 
-Issod oxne SUIWIOD *'1 say sno} nod uoneindod ap ajapour 
uog un sed ang-jnod ys9,u (111) asaujodAy,'T :7"¢ onbavwiay 
“fo no 20 ins 
sogtidoidde woud e suoynqgistp sonsea ap asoddns suoae 
snou ja yuawigtnes ‘J op HomEstiopeu Rl QI9PISUOD SUOAR 
snou ‘Z 39 | Sayapoul say sueq ° 79 aseuUo][HURYSE ,p soUeLI 
-BA B] Inod onb uarq issne ‘g udIssar3az OP SIUDIOIJJIOO So] 
Inod uone[ndod ap sayapou sap Isule SUOAR SNON “SNUUOOUI 
¥ 30 LW sanguiesediodAy sap sane anbruapt uodsej ap ssanquy 
-SIp 39 SoyUepuadgpuUl PWR SoIOJegTe SoTqeIIeA Sop 1UOS 
'y say onb suosoddns snou ‘¢ a[apour 9] sueq :[°¢ anbaveway 


“snuuod 
y 30 0‘ ‘OIULJ SteU opuUIS INoNSUOCT oUN DaAR , Y 
ap a[quiasua-snos un Ins ouLO;TUN UORNgUNSIp aun 


ousiso=p , 9 nN0*°,.N~vYRP QA ~ u(y i)" ae ~U 
(qd ‘0)°N ~ A: sayeurSreut woud e suonnqmsiq — (AN) 


0002 aiqweoap ‘ajenbusa,p senbiuyoe | 


elore/en0 
C- ~~ 


(SI) (WWD py VU 
: d9Av S]Uep 
-uadgpul juos ' sot ‘y opie U oP IAIOSOI SNOS (II1) 
‘7 9[Qpour 9] Inod sumu0> (11) 
‘7 Q[Qpowl 2] Inod surwi0d (1) 


“€ FQPOW 


“snossop-1d sUUOp ¢ s[gpowl Ne ouaUI INb 39 
“0 eseuUoT[NUeYd9 Pp INI9 | 9p SoOURLIRA Sy Ins onb usiq 
Issne ‘gd UOIssaIZ9I Op JUDIOIJJIOO Sa| INS saIOJeETe SI9JJO,p 
so[gpowl sop suosoddns snoy “xnedsatu sinartsnyd yue0d 
-W09 S¥d Ne I[QPOU! Ine] SUOPUd}9 snou ‘UOT}Das aJUesaId 
BR] SUR] ‘So[BUOIS9I soUUdAOUI Sap Inod sIeIq sues SoITegUIT 
SINDJIPSId SINI|[IOW So] ITUS}gO ,pP UlJe sIIOJegTe Indep 
OURLIVA P| Op d[dWIs sJapoul UN gsI[IN UO (766) OY 10 
JOP “OseuUoT[NUueyIE ,p sinad sa] Inod aesaut IneLo,.p 
ooueliea oun gsoddns suoae snou ‘7 a[gpoul a] sueq 


9110} U9[V ANILII,| Ip BULLILA = ¢°Z 


! C ! 
(1) ENGI eee x 
1=4 


ce ! 
—_= ce) 
] (awz- 
3) 


ULIOJ BP] Op Z B[QPOU! Np apes 
3] suep ;9 op Woa\sod ev suudXoU P| Op asITJOMyoe[q-oe8y 
mmayewinse | quoweyes9 suOANON snou ‘sa[gpoul sep 
COA as RI Op 32 sJapour ay UOT}OIIIS BP] Op ona U_ 

i 2 Jed g0e|duias ( -) daAk (Q]) Jed our (n)y4 
so WOLIA}sod & JoURLILA L] OP IND}eUIN)SS TJ * oe red 90r]d 


tal ! tal ! ¢ 
(yd X- A) (gd X- AIT + q 


-WdI ws dace (g) 12 (g) red sguuOp sinareutnse So] JOS 
os d 19 mtd eerie cee sInayeUuINse sa *(/,) 30 (S) onb 


yo ‘g op uoLajsod e souuoXfoul 
sop sonburdure SINIVUINSS So] ‘Z B[Qpoul |] INOg 


“(dix - '4),Cd'x - 'x) x 
'p)D ~ [wh ‘dA |'2] 
‘Ms “T = 1 INdd (AT) 


! 1 eC ‘ 
Luz. 'd) (AZ - as yu + >| 
i le hdialo) an 
I=! I=! 
| [a+ 20,255 40,25] 
ic ul ub 
I=! 
. d+ ZU 23} ~2O9dUlAl a 
a 


C(O + XX) ZU + 4, XD) 
_ (gre x'D))N pin [20 4419] 
‘we p=rimog (0) 


¢ ! Jy 
a = 
I q 


‘ 


‘red saguuop 

JUOS Z B[QPOU Np sped 9] suBp sqqIy op aseuUOT[HURYOS 
un nod sargjduos SO[[SUUOTIPUOD sUOTINqIISIP say] 

'y sop douRssteuuOd on8eA oun JoIg]Jol op ulye 

sijed san sIstoys JUdWATeULIOU snuUOD sanguIerediadAy sop 

juos ‘g 10 'p no ‘('¢'v) 5 woud e suonnqmsip sap quayod 

-W109 39 sjuepusdgput sasoddns juos ‘2 soq :7°7 anbaeway 


"SqqID op oseuuo][N 
-UbYD9,[ BP SIAN Ud VSTWI eB] INod ayno1JJIp ap snjd uou sed 
v A.U [J ‘IOS Op BA B[RBQUI INaLIO Pp SOULLIA BI ap s[Epou! ne 
Q[&39 INDI Pp VULVA LI Op o[Qpou np uoTsua}xa,| ‘sokeg 
ap onbryosese1y ona op jutod un,q ‘2 say sno) nod 79 = me 
onbsio] | e[Qpoul ne yinpel es Z aapoul vq :[°7 enbieway 


snuuoo y 9 0“g “Dg 19 | -©=U'‘,'9 pie 


Rony HGR So Vecen op ae (q! 'D)D pur ? 
‘(qd ‘0)? N~A ‘sofeulsreur woud e suonnqiysiq (It) 


(€1) (mT = 1) (@'hZ)'N~@‘h|'g 
DAB 
sjuepuadgput quos ’g say ‘@ 30 A aonb aasasai snos (11) 
ne ‘T= [iu 6. | =1) 
(ZI) lo dq fi cen 
q'x)N~{o git 
Dav sjUep 
-uodgpur yuos “4 soy ‘ ,0 yo ‘g aonb oasosgi snos (1) 
‘7 F2POW 


S1IOIOA onb o][eZOUI INILIO Pp DOULLIVA 
RB] op xneoalu sinoisnyid e onbryssessry ustsaheq s[Qpowl ne 
QUIUL | S[QPOU! Np 919dIIP UOISUd}X9 SUL) “UOIZ9I OWIII-? PT 
Inod sseuuo][HuULYIE ,p INDLIS | Op DOULLICA SIRIA FI 2 HOS 
‘QSRUUO][NULYIO Pp SINILIS soy Inod ajesaut IndLe P ooueld 
-CA OUN SINSUIpe ._p ajsi[eas snjd jso [I ‘anbyeid ey sueq 


Q[VSIUI ANIAII,| OP ULLIBA 77 


‘Z’€ UNIS BJ & I[QPOU NP UOTJET9S BI INS s[IeJap sop 
RIDANOI UC ‘a[QPoul Np UoNosI9s B] B jUoWIONbiUN Yes oseU 
-UOT[HULYDE Pp DoULLIVA LI Ap [Noed 9g] ‘safeuoIses souUsAOUL 
sop UOTeUITISE,| B NOLNS suOssaIgjUI SnoU snou onbsIng 


; I=! 7 
(TT) ; [rw Foe | 


I- 


an 9) 


aaa ane? 
) 


QULIOJ BP] SNOSs JUaTNGO,s 29 op LOLA} 
-sod v QUUdKOUL FE] Op asT][aMYAoe[ -ovY InoyeulNsa J ‘asIOA 
-Ul vues EOBRGUISIP aun ayodui05 79 uoWaTjouuON 
-Ipuos ‘anbsing 9 aseuuo][nueyo9 panoua, ] op ooueLea 
eB] @ UONeUISE,p cinpgooid ourguI Be] Jonbiydde yned uO 


(on) XH X= (M4 


ifr 
or PEO RU i ad 


suoi6es seyjed snod souushow sep enbiyoiesaly euualseAeq UOI}EUIIS :OBY Jo NOA 002 


ap spre] & I9UIN}sa ssoye 3So ‘Ml sfeuOI3aI oUUaAOU PI 

©) LOLIa}sod & soueLIvA BT [YW oxouue | JOA {{ 5“ ‘T= ¥ 
‘oy? "MO “qn oT = ‘od } sqqly op suo][HURYde sop 

opie | & yUoNqO;s ‘( “'d) A 910u “g ap WoLaysod ev aouvlea 
B] Op gsljjamyoupg-oey Inajeunss,, ‘(6) ep opre.] Vv 


(6) (C2 ‘wy ‘A ‘4 |‘) - 
(Crh a |'DaT+ (Cv Z|'9)A)7 = 
(Creyh ge |'D A + (2A A/D) A) T= (Al'DA 


anb sajou y “xX (4|'9)4 X= (| A 
anbsind “dg op uorajsod e soueLea ek] ploge,p suoanon 


snou “n ap uoajsod e souelea eT Ius}qo.p Uy 
"(€ UOTIDES 
RI MOA) UONLSITTAaMYov[g-oey ke] sp siojjo sop uoneny 
-CA9,] OP ONA Ud SINoJeUINSS xNep sso ep UONe[NUIs op 
sodA}-sInoLe Sat JoIedULOD ap 39 Ja[N[Vd op JUSSIIQIUI 489 [I 
‘stojainoy, ‘sayjamouod suonewinse sourgui say onbsoid xnap 
sno} jusUUOp Gaul of) ou aonb 99 & suopuaye snou SnON 


(g) CoghZqO + 4 Xa) 
fad +X. Keg tae andeX = aol! 
yo 
i tat) py) ! ! ! 
(L) dX ST = @I 1X = @t 


yINs SUIWIOD sguUOp JuOs ‘1 a[eUOT3 
-91 suUdAOU Be] Inod syuepuodsaLIOD sInayeUINsS so] 
(1661 UIUS 19 pues[ey) 
uonejnuis ap odAj-Inaua | ap 3sa inb 90 snod od onb 
ANITIIOU! 489, no SIOJaINO TL, ‘WOLa}sod ve suudKoul P| Inod 
seq sues xnap. Sno} 1U0S (ep i i) nd Peis cute ory 
ul 


. InsUUNSa,| 189 (Joan dw Thereutnse 1382 | ad no 

(9) “CkZ@pd + A, Xo") 

; . 1=1b) 

CAO) a XX ) ST = 
I-45 ; 
(ay? ‘@O ‘yh ‘41'9) ST = awd 
jo 
, [Neale g) : 
(s) on KG = 


:‘g nod snossap-t9 sinayeuNss sorne xnap say IsuTe SUOAR 
SNON “(1661 ‘0661 YS 19 purs[aH) gstTfomyou[g-ory 
JURSIP-I0S INa}eUITISe,| 3So9 yUepUodsaIOD IN|}eWINS,| 
19 ‘[[JOMYOR[ G-oeY op NUUOS UdIq SUIQIO9Y} 2] INS spuoys 
as uoneoyIpow ane “((*2 ‘7 ‘AL |'d)7) PASC |'G)eA 
9 “(72% ‘A4d | ‘DF = (4 | 'd)7 anbstnd ‘uoneumse 
anou seroyoue.p ue [2 yh yh X | ‘IIT } 


661 


Sa][euUOTIpUoD saoueIedsa sap uOT[HURYDE,p suUUeAOUT 
eR] Jostjyn suoanod snou ‘aguuey suo} op aja {duros 
g][aUUONIpuos uoNNqMIsIp aun ayoduioo ‘g SASL 
{, pos np uolHueyse,p suueAour ve] JastIqHN yned uo ‘'g 
op Touaisod e eee R] op inayewnso un puree Inog 
‘nuaigo wos {D* ‘T= 4% y ah anO) sank eT = iy yo) 
a0} Rl op ‘ 3) au 2p uojjaueyse un nb suosoddng 
"1 99 By ‘ASST =1“g ap sgsoduios uowaysod ® uOTING 
pau R] op sted ' D ee sop swmu0s {5 * ‘, =¥ 
‘Gene SqenO “Genk Me‘ yo} oe «=uo-— sind 
‘Q0U9S8JOAUOD BP] & nbsnf Poa op op spouted ‘7 suosip 
‘saldA0 ap 9AdTP SIQUIOU UN 9jNd9X9 UG ‘aseUuUOT[HULYIE ,.p 
ajoAo oun see ep juoyouuiod (p)-(e) sade 


SSO ey one pd Poprere ls 'd a2) 
op med e 7 «Jam ap (p “ 1 qyh‘uo T=) ag 
op epreal ze tr by ‘f‘aley] op sned e ® Jol op 


(9° ‘oy YP MO Mo "T =! qe (Pepe Te v 1 ‘g 4A] 
op ined R (p> Jon op (q a U, err | a 
ied e ‘a’ ‘T= 1° au JOIN op We ,s [I ‘ ot  @U ‘Ah 
yedgp op sinojea op opie] Y (e :1Ins gumu09 ajuesaid 
QS SqqID op o8euUOTTHURYDE,p epoyjsu eT] ‘suOTTHURYDE 
Sop o1inpoid op jUSWIaTIORJ JUSVOULIOd 19 SOULIO} OULIOJ Op 
JUOS Sa}Q[dWI0d saT]JaUUONIPUOD SUOTINGMISIP So] SajnoL, 


Lal ! ta! ! =i) © 

((dx- 7 ORG: Ga? 0 ls at 
he fe ¢ 4 
We ee lo-tetdal tl aD, 


[suze ae owe | 


“paling exo ew) 


i buannleaeyiA PU Mee er sere) 


ded * 20,2! ‘us)) NU ‘od 4alAl = @® 


(+x) Ez0+ 14 XD) 
+X XD)N pay (2D 41'S) 
ws T=rinog — (¥) 


‘Jed seguuop saygyduros 
SOT[OUUOTIPUOD SUOTINQLISIP So] INeJ snou [I ‘[ s[epoul 
NP oped oJ suUBP SqqID op sseuUO][NHUeYD9 | Jonbydde inog 
‘sagsodwiod WoLa\sod & suolNnquysip sep ited e suol[y 
-uvyog sop eimnpoud op ulye (0661 YIU 39 pueyJoD) sqqrH 
op sseUuUO][HULYDE Pp BpoyJoU! ke] SUOSITN snou ‘UaNbes 
-U09 Jeg ‘a]qusivj sed jso,U [NITed J] 39 ‘a[[OUUOISUSUIP SQ} 
onbusuinu uoness9jul oun ayodui05 egsoduos uoLa)sod 
B UONNGINSIP B[ ep asamp woNeN[eAg,T ‘gd ap suoneu 
-s9 sop yuapuedep inb ‘ ii X= ‘n sayeuorsei souuaAoul 
sop Honeied eB suoneurlsa So] JoAnon ‘“atpnorsed 
Ua'se.('as [= (ate pea HAY) = A saguuop say inod 
‘g sop UoLa}sod & suoNNqusip sa] JOANON SUO[NOA SNON 


000z e:quuadagp ‘ajenbua,p senbiuyooe | 


“QMOJVZ[L INILIS,.P DOULLIVA ap Jo aTeSouI 
INILA,P POULLIVA BP So[QPOU! sop suUOIINPONUI snou ‘¢°z 
19 ZZ SUOTSAS So] SULC] ‘SOJUDIOJJIP JUSWETqQISsod ImaLa .p 
SQOURLIBA Sop aINOUIPe WeIASp [eigUEs snd s[Eepowl up 
“SUOIS9I SSSIOAIP Inod saqusIOJJIP og JUaTeNod aseuUOT 
-[HUBYd9 _P Ina] 9p S9OULLIVA Soy ‘StOJoyno} ‘onbyed ey 
sued “SUOIS9I Sa] $ajno} Inod ° 72 g[e39 INDI | op soueLIvA 
gun suosoddns snou ‘| 3]apour gI sued :¢°, onbavwey 


“saguuop sop 
asA[eUe | INS € UOTDES BI B STUINOJ JOOS s[IvIep sag ‘uOUd 
B SUOIINGLSIP Sop ONSA Zasse DOURSSTeUUOD UN Jo}9IJoI & 
UOSeJ Op SISIOYD so (samQuIeIediadAy say) Woud ve suonnquy 
-SIp Sop songuieied sop Inge eT ‘soguidoiddeur wota)sod 
% SUOIINGLISIP op souasgid ev] sIpuleso e sed suOAR,U snoU 
‘ISUTY “(966 BI[eSeD 10 USqoH) sogudoidde juos WoLIa}sod 
B SUONNGLISIp Soy soyno} onb JoInsse snou op ulye snuuOoUT 
songuiesed sa] sno} ns saguidoidde woud e suonnqmsip sop 
9sI[NN suoae snou onb J9}0u ap syodull |] 37°] enbaeway 


“souUdISaARg SOPOYIW Sop UOTIEIMUT] 
oun & 3S9,.-d ‘o[qIsne{d yeprpued uN so s[Qpoul op saTeul 
-ered so] ins s9uidoidde woud e uolnqiisip 93n0} anbsind 
g[durts oyoe) oun sed jso,u Wold & suoTINqINSIp op x1oyo 
3] ‘ouUsIsgheq soUsIOJUI oUN Inod ‘TeI9Ua3 U_ “LIOLIA)sod 
B UONPUINSS,| JoJOI[gWIe RB DUOP 39 ‘LOLIa}sod ev soouel 
-RA Sop 19 LOLIa}sod e& sauUdAOU sap sSasI[[amyoe] g-ovy 
SINDJCUINSS Sd] JOANON & IIAIOS JUSANId VgULIOJ SULIOJ 
Op sajq[dui09 sayjouuoNIpuos suolNqMsIp sep (Z) ‘asinb 
-O1 BIOS sgUUOTDaJI0d snid oMOJegTe UOTeIQUS OUN ‘sed 
99 suep ‘sooLesiyeuLOU sajueJsuOD xne nbsnf snuuod vies 
enbuozjenb aneuiesed un Inod aja[duros ayjouUONIpuos uO} 
-NqLysip ev] ‘uostesnfuos ap souasqe | ua ‘sqqId op aseuUOC] 
-[nueyo9g | op odej9,| @ ([) :JIns suTWIOD 99}10[dxa ang ynod 
uostesn{uod ve] ap soUueLOdUII TJ ‘saltoyegTe S}OJJo,P ISIOA 
-UI SOOULLIVAOD-SOOUPLIVA Op SOLYLU e] INod euWes UOTNG 
-LSIP SUN Pp So[qeUeA SInatsnid & UOISIIA PJ ISO WRUSTAA OP 
UOINGLYSIP eT “USWIdAOEdSal LeYSTAA Op 39 CUTS ‘aTeU 
-10U SUONNGLISIP sap SIOJ 9UN aI0DUd JUOS INb 75 39 72 ‘A 
Inod sajg{dui0d sayJouUONIpUOS suOTINGIsIp sop B JUSUSUI 
soja,nb suas 29 ua sagn3nfuos yuos (p) sed yo (¢) sed 
soguuop uone[ndod ap jo a8euUOT[NULYDE ,p SUONNGUISIp Soy 
30 (11) odejy9.] e UOLd & suoNNGUsIp soy] :][°T enbaeway 


u = (0)"] 
(T= ne 


(ené.e}s ate 


SUIWUOD SIULJOp SITqeLUBA SINIIS 
-njd e vues UOTOUOJ oUN js9 (0)4 1P0<Y¥0<X 00 


(Fey zdoC 


Z 
‘{(yy) 0 axe e LY| 
I D 


gIsuap op onens QUILUOD SOAR tt M op uoTINGLyAsSIp oun 
9 (y ‘D) “4 PO<xX‘0<4 0 <2 ‘oe Fy X(P) Mn T= OF 


Jed o9guuop 9sisuep oun seAe vues UoNNqsIp 
oun sUusIsep (q‘P)D ‘1 e[Qpow np (Im) ada, VY 


‘snuuod y 12 0 ‘q‘D ‘ap 

1-O = J (70="1 00 (yD) ~U PG ‘D)D~"2 

(d‘0) N~A :soyeursrew oud e suonnqmsiq (Itt) 
:JINS SUIUUOD 

sanguieied sop Inod youd ev suonnqmsip sat suoidepe snou 

‘uaIsgheq 3[Qpoul op uoTduOsep aNou Ja}g[dwi0o ap uly 


(v) (mT = 1) (@'KZ)'N~@ 4" 
DOAB 
sjuepuadgput quos ‘g sal ‘@ 10 A aonb oasosgu snos (11) 
‘ 'y Ce a (su Ss ae 
(©) ( lie T=!) 
“( 0 “gM N~20 “J "4 
dav SJUep 
-uadgpul qwuos ° ne soy ‘ 79 yo 'g aonb oarasol snos (1) 
*T 9QPOW] 


"INS SUWUOD xneaalu sInarsnid eB (Z) efapow np anbryo 
-IeJgly ouUsIsXeqg UONeJUASIdeI oUN SUOJEPISUOD SNON 


9[@S39 ANIAII,| IP VULLIVA [°Z 


Sd419 Ad 
QOVNNOTILLINVHOG.T Fd AONGYAANI 
La XNVGAIN SUNGISNTd VATAGOW “7 


*SUOISN[IUOD sou yo sonbseur 
-o1 sop suojuasgid snou ‘p uOT}eS eB] B ‘UTJU “[ISoIg np 
(SUOIZ9I) S9}UIOD OP SO9BCUDUI SOP NUDAAL J] INS saguUOPp op 
Opie] & S[Iqisuas el op osAjeur | 19 BQpOu! Np UOT}DdIaS P| 
SUOpIOGE SNOU 39 SpOUIEU aI}OU SUONSNI]I snou ‘¢ UOT}DeS 
RI Y “exouUod sqqiH ep eseuUoT[HURYDg,| Op PDUEIJUT | 
ap 12 INdLIO,] Bp soueLIeA e] op Isa Inb 90 nod sasayjodAy 
SOSIOAIP DOA XNVAAIU SINaIsnid e sonbryosesgry sustssheq 
Sd]QPOUW! So] SuOJUSsaId snou ‘7 UOTDS BI YW ‘S9XdUU0D LOLI 
-a1sod B sooueLeA say 19 sonbryaressty souuatseXeq suoneu 
-I]S9 So] ITU9}g0 Pp UlJe sqqIy op eseuUOT[NULYDE .p spo) 
-9UI 8] SUOSTTNN SNON *xNesAlU sInotsn{d eB sJepour Np sanQui 
-erediadAy xne agioosse oireyuowigtddns opnyiisoul.| op 
quowanbryewojne 3}dw109 Just) WOLIs}sod v soURLIRA BT “WOU 
-91S0d & QOURLIBA BS OP UONDUOJ US B9INSOUI So UOISIOgId 
BS 39 ‘MoLIa}sod v suUdXOUL Bs ap apie | & SgUIIYSe Yso ‘Tl oTeU 
-O1391 BUUSAOU VT “SOXIJ a1OJeg[e INSLI,| Op SAOURLIBA Sap 
NO 9[eZouI InoL9,| op ssoueLeA Sop jUs}owpe Inb xnei9U 
-93 snjd xneaalu sinotsnyd & So[gpoul Sap % suOpUsl9 | snoU 
yo sokeg op onbryosesory oped un UOTes xneoaru sinotsn{d 
B (Z) a[QpOUr 9] sUOUTUeXe snou ‘gsodxe juasgid 9] sue] 
"SagSI[PPOU! J[BUOIS9I UONLUINISS .p SOPpoyIsUI Sop 
gteiep ndsode un guuop 1UO (6661) OPA 19 (F661) OFA 19 
YSOYDH ‘sellOQuId INdLIO B UOISSIIS9I OP Sa[QpOul ap apre | & 
snuajqo sonbuidwe gs TdW sinsyuwiNss xne JUSWAATe[AI 
anbuiduia gS Td dAi op sinayeunse sap gy1ovotJa.| op 
JUSWIASSIOINOV UN NUI}GO jUO SII ‘[ISsIg Np (SUOTS9I) SILOS 


suoibes sayjed unod seuueAow sap enbiyosesgiy auuaiseAeq UES :OY Jo NOA R61 


‘OUS STM epeuey (oueUC) EMENO ‘UO}E1FeD sIISIZATUA ‘sonsHEIg pue 


soneulayyey JO [ooyds ‘Oey “N'f {9.LO WLM epeues (ouejuQ) emengC ‘epeuea onbysneis ‘soseugui sop saidne sajgnbue sap sapo}9Ul sop UOISIAIG ‘NOX BuO 


OP SOSLUDUI SOP NUSAAI 9] INS sagUUOP op spre | Y “XNedAIU 
sInaisnid e@ sjepour a] Inod onbiidwse gsTqW np (INO) 
ouusXoul onbryeipenb ino1a,| op xnop sip1o .p uoTeutxoid 
-de oun ja onbuidwia qs TqW 3] Nusjgo JUSWIaTeB9 UO ST] 
“UOIZ9I DUIVI-1 BT Inod sorelIxne sojqeuea sop uoNeindod ey] 
ep sanuuod souusXoul sap [ x d INd}99A 9I 189 ’ “X QO ‘purrs 
‘N yuesoddns us | 'g) 'y = ‘1 feuorsai ouuaXoul | op (GS Td) 
steiq sues ameguly Inayorpaid Ina[[law a] IWusigo e UOdey 
op snssop-1d xnedalu sinaisn{d e a[Qpoul 9s] aIIej ud Inod 
(0661) O8Y 39 peseig Jed a\dope aipeo 9] IZIeTS UO (6661) 
WOH 39 VINOP, 19 (C661) PINOP 19 YOH ‘e[epour ourgui un 
Ud UOIS9I BI OP NVIAIU Ne 39 SHUN, | Op NVIAIU Ne Sa[qeURAOD 
SOP B SINOIII J] JUDUIOATIODJJO oISQ\UI (Z) BJepow a] aonb 
I9}OU Y “ONUUODUT ISO q SSOULLIVAOD-SdULLIVA Op SOLIRUL 
eno ‘M = (‘4)4 10 C= ('4)q D0Av UOIZaI oNbeYD ap ures 
ne sgsoduios uoNNqsIp eun JUSLOdWIOD 39 aIyNe | & UOT 
-91 9UN,p syuepuadgput juos ‘a sa] ‘uOIs9I ulal- -1 P| vee 
sollojyeg[e S}ojjo,p | xd In9}00A uN js e(7 (ae mA) 2 = 

19 SOXIJ SJUSIOIJJO09 op | x H 1n9}99A uN 19 A ‘uolsgl P| ie 
Nvaalu ne sofqeuea sop b x d ueyd np so1neuw ej so ‘7 no 


(Z) fat k'7='f4a+'d'v= "7 


:JINS DUIWIOD JoNUOUD aSSIR] 9S BTQP 

-OUl YJ “SUOIS9I So] UD SUOTeLIRA Sop sted oun Jonbiyd 

-X9.p UlJe UOIZEI P| Op NIAIU Ne SoATJeoT[dxa Sa[qeLeA sep 

B JULYeeI SI] US 39 SOIOJYTe UOISSIIB9I BP SJUDIOIJJI09 

Sop JULSINPONUI Ud XNedAIU SINaIsN{d & s[epoul UN sITej US 
Inod snssap-1d a1ped 9] ISIe[9 1U0 (€66 1), INOW, 19 10H 

sake Pp ak 2% 

‘u Inaen3uo] ap guun ce 9] 189 uf Lows Leda 

souyeUI oun sd He ee han SY WC ii ‘{ no 


“es diy 4 


‘JINS SUTWIOD aT[OIOLNeU UOTe}OU oUN ,p uO 
-OUOJ US JoWILIdxe asste] as (1) B[Qpoul Fy] “ayeUOIZeI UOTE] 
-ndod vy ap arte ey ‘'ay red goujduras ‘wv saav onbrdde;s (1) 
gJQpoul 9] ‘uoNe[Nndod x] op ajquiasud | nog *sjuepuedgpul 
sgsoddns juswisje89 yuos “a 39 "a “%0 = (‘ayy eQ=("2)q 
DOAR SaIULpUsdgpUI sIIOJegTe INIA] Bp salqeuea sop 
juos “a soj anb suosoddns snon “fo = (aly 19 QO = (a) 7 
dOav sjuepusdgpul xneUOIZe1 s}oyJo sop juos "4 ‘soxl 
UOISSaIZal op Saneuesed d ap s]quiasua uN jso g ‘9I1UN | op 
neaalu ne TUG So]qeLIvA sap | x d INd}99A 9] aJUaS 


“UM Gees “Tl = 1°'9 


-gidar "x ‘wi ‘] = 1 ‘uorZar outet-2 eT op seguUOT[HUeYDE 
sgulun xne sagioosse suoneasasqo sa] sjuessidar “ no 
(1) me Ta 1 lu p= fia +a + Aer ah 

JIns SUIUIOD 


auaseid 9s J[QpOU! J] “SdgsI[SPOW SaTeUOIsEI SUOTIEUIT}SO 
SOP I1U9}QO _P UlJe JOYOUS INDIO B UOISSaIZ9I Op sJQpow 
un gnbriyjdde 39 gsodoid yuo (9861) Jal[NJ 39 JouRH ‘asoneg 
“SPSI[QPOUI S}OOIIPUI SINIJBUINSS Sop JUSUTWUe}OU ‘saxoUUO0D 
SUOIS91 Sop ins juaIndde,s Inb sjooJIpUI sINoyeUINSe sop 
JOSI[NN Pp slessaogu DUOP 3S9 [J ‘AapueIS Zasse JUSTE ISO 
XNVUOIZ9I SUOT]HURYIE Sop aT]Ie} B] onbsind ayenbepe uorsto 
-91d oun sed juassiuinoj ou UOIS91 oUN eB saldoid sjouUOnIp 
-B1] SJOIIIP SINDJCUINSS SI] “SITQeIJ XNCUOIZ9I SINDJeUINSA _p 
oN199e OpULUAp BI Sp dsned & B[BUOIZEI UONPUINSS | B UOT) 
-uoyje .p dnooneag apiosoe uo ‘saguue senbyenb sindeq 


NOILONGOUINI ‘T 


“UOIB9I ‘aseUUO] [NUDE _p 


INGd,| 9p oueLBA ‘xneaAlu sinaisnid e a[Qpoul ‘sakeg op onbryomesgiy ‘sqqig op oseuuoT[NUeYyY :Sq1O SLOW 


‘[IS21g Np (suOIZes) soiWI09 sanbjanb ap saseugu sop NUSADI 9] 


INS S9QUUOP 9p opre | & FaNsNIII Iso aINpgo0Id eT ‘gII]IqQIsuas eI ap asATeuR | Jo UdISeAeq o[QpoOU! UN Pp UONeI~s ET JUSUETeS9 
IIPNI9 UOC ‘O]JED UOJ] Op SIN soy JURSINPSI SasT[[OMyYoe[g-OeY SINaJeuINse sap JUSUUIGO S|] ‘sofeuOIsgI souUdAOU! Op 


uOL9}sod e JoUeLRA P| 39 WOUa}sod e& souuaAOUI Sa] Japnoyeo Inod sqqiy sp sseuuo][NUeYoE 


([ Juasi[nn sinoyne say ‘xnesAtu 


sinoisnid e s[gpow np saygurerediodAy xne og100sse osrejuowlg{ddns apnytyeout | op juowionbyewoyne a}dui09 juan aq 
“UOTJLUINSA | Op UOISIOgId eB] OP INSOUI Op YOs o[eUOIsE1 DUUDAOUL PE] Op WOLA\sod e soURLIRA BT ‘WOLA\sod ev ouuaXoul es 
OP UONDUOJ Ud d[BUOIS9I QUUDAOU OUN JUDUIT}SO S]I 19 Soke op onbiyoIeJgIY SIP UN ,P UOTJOUOJ Ud saTEpou $90 yUaJUaSeId 
sInoyne So] ‘Inde | op soueUeA ke] Inod onb uolssei39I op saneuresed say inod jue) sasojegye sjajjo & soTepoul sap juesoddns 
Ud NO INILI,| 9p o[eSguI DOULA SUN JULIIUIPe US Sa[QPOUI Sa] JUSUIOTeBE JUASSISILTN sINd\Ne sa] ‘soTeUOIS9I sUOTeUINSS 
So] Inod soo0jyeg[e UOIssoIs91 op soNQweIed sop jueyoduoD xnesAtu sInatsnid e psepULys so[Qpoul so] oUTUIeXa UOC 


aWwnsay 


;OVa MN'f #9 NOA DNOA 


XNVIAIU SANIISH{d & Sa]VpoOul ap IpIE | & 
suoisa.4 sajyed anod ssuusAow sop onbiyo1e191y suUsIsgAeq UOneWS| 


Z6l 


epeued enbysiers 
902-261 ‘dd ‘Z ,U ‘9 “1OA 
000Z eiqweosp ‘ayenbue,p Senbiuyoe ia 


: art - EY , " i fy Fi 
_ f f Pe 
: res 7 a 
ate Pq st, ce 
e% cad eh : a 
ess 
a 
nee e | 7 
* = A ’ 
ii 
1 
7 = a 
e% t ! 5 
s - , 
cy 
r 
~ j\ 
Fe 
¢ 
rai = 
a Adis e “, 
i r‘ vy ry berry ale Ret lade bal P ms { 7 ; 
‘ ya vey plc AS fms “+ ; nyt 
iio Wie Ion ecg te Gitigls co ge 
; ' Gre OY oe : Mt 
« a \'S at 
oa elie , 
( ¢ ri eh ( bm: ae} eta rte , _ } « 
aah OAe s i ’ a as bl aaibe ne ay . 
r AL ery , Pe oul ts fod ant a3 ; ; 
i a ‘ : ‘ ; 3 s F 
eae 
} 7 «l 
men's on aD . ; : ~ o Ooart 
¥ a le hint tae yy 7 aT eyes.” | re 5 1° Ca - ai ai t ‘ 
ero werd eat se" ghee | 


i 


| oi bs at | : ; bie A Ua , | ‘ 


a pana ayes A ho ; a ui i Shes 
WL LOL RECEP 2 ; ee : - ’ ATS = 
y 4 
‘ > f 4 
i 
rata teresa A.sho22i 4 | 
i buh 
poh ii cy rs Seed ** | ; ; 
rig aa u 
ne fife ‘il ne ' on! ad 


] * i 
ot ao i 


eae i* ‘a “t ot 7 | 

Sepia, th, WTneal ie Ay ee bt Las ‘he y ary ” 

tty ENS, ! wi i? sito Pra Iere wai Tan oe) ¢ 5 | 

rey * aie wh iat Phare. ), My ba ha tit 3 . AA 

MW. 5 prem oe waar ; ¢ a’ Pes 5 ; 

pe en vi 7 Ae Seki ta ia! “eV BT! (es * ‘ a av ’ 
earth porch (aaa ne} el REC ey Tk RR 

pag eho . i ¢ uth ies eT wtp YY “ % : fh A » 

r dpe ee SPR, aT : | wc te! eto 


A Sen 


ice, ; ST ia” tf ae De gaahl ocak Ua, : 


ey ae te 2d \e une, ~ 


eiahy D Wega We UEAN yr ty, it ool ee?) i ty’ ty GOP mde ¢ ¥ A vegul) C4" 1 


sh ene ALL ; , ie ie i ae mer 
te ater ate ie o ae ee Mine ee | 
; Shh est Bilge T it é at rire) "1 ne - ey i ee. i? \ : . i 
niodliacalieh bans PE I Ce ee, : apie EVs idee coe ee ; i i 
fet ye HR id Mee 7 cs ac AL ae. tha Sven), Me 
5 CPP PTS, COE (oe oe CAAA, 1 aS fiiiher 4h " Pe Mipie eae \ Wie ee 
aside — peatrenisee quid re ee gato yy hae on DATS Seba ier Aik ar ie. ee SRS: 
a 5 | 


we a 
re 
late ie 

ve 


eae 0 = 8 te eae te a. 
aa | Ly A ; - ae as nae, & ; 


os 


ane 


*€Q ‘UOIJDIIOSSP J0I1]S1DIS 
upr1daup ay) fo JouAnor ‘spoysJ JUoW sN[py JeuOsesS STg 
pue snsudad oy) 0) suonewtxoiddy seoury (8961) ‘H'V ‘OSNOQNOA 


“LOP-LOE “OK MON “Ssoig onMapeoy 

‘(SPq ‘ORY MNS 19 ANI “Y ‘PsMery “q) ‘Susjdiung koaing 

ut soidoy juasin ) ‘solias poziyeuoseesap & JO} UONBUIT}S DOURLIeA 
Jo walgoid oy) UD (1861) ‘f'N ‘UNOSNOW 1° “Wt ‘AALIOM 

“C8-bL ‘Spl ‘¥ A1a1009 poousuvig 

jokoy ayj fo jouanor "pouyjoul-[ [X IY} JOJ sJoj[Y JeoUT] :eIep 
JUILINI JO UOISIADI pue juoUT}sNf[pe [euosedS (7861) “AM ‘SITIVM 

‘O91-6r1 “8 ‘SouSUIDIS 

porno fo jouanor “Kaaing uonetndog juaND ‘s'Q 24) Woy 
eyep AdAIns o[dwes Jo Zuljapoul sates OWL], (7661) “AU ‘MATILL 


C661 [OISUg “All[eNnd ssad01g pue JUSUOINseA 

AQAING JO soUdIZJUOD 4, e gjUasgid spINIW ‘UsISOg 
o[duieg pur sisjeuy [euoseag “($661) ‘D “AAT 1° ““V “AHAITIOLNS 

‘ongoyjejeo ne [GE] OJouINU 

‘sonsieig Jo neoing uelensny ‘Z/¢6 O1ouINU ‘sonsneis porddy 

pue soljowouoog suep [eae] op UOWINDOG ‘suoLy Suijduoy 
pup uoipsoduosag satsagy awit [[-X “(€661) ‘V “AdAITOLAS 

"eLIaqueD ‘ssaIBUOD sadUdINS [eoNeWoYIe [UOEN 

RL eR gluasaid ajay ‘souag jeonsneis Jo Aqpiqeue A oy} 0} Joug 
surdureg Jo vorNnqMUOD ey (8861) “Y “THIN 3° “O'd “THALS 

9L-19 ‘OT ‘sousumis pero 

fO [DUANOL *S2JCUIIJS2 PUD] JO SUOISIASDI 9Y} UO suJONed UOT}EI0I 
JUDIOJJIP JO 199JJ9 BY, (0007) "HD ‘NAUVTION 1° “O'd “TAALS 

‘Ob-S “O9I ‘F iat20g jwousiwig jodoy ays fo jouanor “uonTUYep 

ddJO INOQe] [BUOTVUID}UT DY} 0} BuIpIOOOe yUoWAO;dUIS pue 
juowAojdursun Jo sayeunse AyyUOW SuIONpOlg (L661) ‘D'd “THALS 

“99030 Teonsneig 

[enusg ‘uonIUyeq OTL ey) 0} SuUIpIooDy jusWAo;duIoUQ, 
Jo soyeumnsg ATyUO Suronporg 10¥ suondO *(9661) ‘D'd “THALS 

"PE-EE ‘O9T ‘V Mta1209 joousunig jodoy 
ay) fo jpusnor “[2a\§ kq Jaded Jo uorssnostq (L661) ‘A'W'L ‘HLINS 

‘epeuesy onbysiyeig ‘ansojejeo ne 97S-[, OJQUINN 

‘epeued np sanoe uonendod e] ins ajygnbua,| op aIZ0opouje 
(0661) ‘d “VGAVW 1°" ‘ONISINVD “C'f ‘MAU “dW ‘HONIS 


S6L 


‘O'q ‘UO SUTYsE AA ‘2dIOUTUIOD 
Jo yuowrpredaq ‘S'fQ ‘snsuad ou} Jo nesing ‘¢]{ anbruyse) sjonsy 
‘UDABOAd Juauysnipy joUOSDaG J] poyjayy snsuaD ays fo juv14D4 
II-X (L961) O'f “AAVUADSNW ? “HV “ONNOA “f ‘NIDXISIHS 


"EL-€ ‘Sp ‘anbys1jyjg ap ajouoljpusajuy anaay “SKOAINS 
poyeodar Jo siskyeue oy) 0} spoyjoW soles ow) Jo uoNeotdde 
UL (LL6I1) ‘D “A ‘SANOf  “A'W'L HLIWS “f£V “LLOOS 


‘60S-Z6P °69 “UoUDID0SSp [DIISIIDIS 
updidaup ayj fo JouAnor ‘suotsed90 payeodal uo Burjdwes 


Joy susisop uonrioy (p961) “AL ‘NVHVUD 1° “WNT ‘OVY 


‘OLI-S8 ‘SI ‘sisdpoup sataag 
auy fo jpusnor “sloyeulnsa paysn{pe ATTeuosesas | [-X JO SoOURUeA 


oy) ZupeUINse Joy poyrouw! JesoUes Y “(/661) ‘dC ‘NNVWadd dd 


“SZ8-S08 ‘ZZ ‘VW ‘SOUSIIDIS UL SuOIjDIIUnNUWMOD “BuI{dUIeS 
UOTIEIOI JO JOMS oY) 0} yOalqns sotIas ou e Jo sIojoureIed 
oy) jo uonewnsg (€661) “AOD “WANOd  “S'd ‘THIVZVIN 


‘C6L-06L ‘L661 “UONDID0SSH [DIIISIIDIg UDI14aUy 
‘spoyjayy Yyoavasay Aaaing uo uo1jsag ays fo s8u1paaz04g “SIOY 
puay} pue jeuoseas Jo soueLeA Bul[duies ay} UO suId}ed UOT}e}01 
WIIJJIP JO 199}J9 SUL (L661) ‘O'd “TAALS 2° “H'D ‘NEYV TION 


“SUOSUOTIOAA JO AjISIOATUE) 

‘sonsyeig poyddy pue sonemouiey, Jo jooyos ‘je10}00p 

ap eseu, eettqnd uoN ‘sKoaIng payeodoy ul UoNeUINSY pudal], JO} 
SIoy[Lf pue suloneg uoNeoY SulUsIseq (6661) H'O ‘NAUVION 

“angoyeyeo ne Q'QSE] OQuINU ‘sonSneIS 

Jo neoing uelensny ‘s“ojvo1puy a1 0U0Iq uDYDAISnP ‘“KULOUND 
ay) ut sjurod suruim surlg (1661) “f “AUVZ 19 “S “AYOWNIT 

‘Q0€-L67 ‘91 ‘algnbua, p 

sanbiuysay ‘epeued np sayoe uonejndod ey ins aygnbua | inod 
Joued ap uoNe[al09 ap sjusToyJa00 sop uoNeUINsY (0661) ‘H “AdT 


‘prO OW “MN ‘uorstatq Aiypend 
pue spoyjeyy ‘sonsneis [eUOneN JO} 9ONJO ‘spoyjoy] UONeWIns|” 
pusly, Jo uonesnsoauy UY (L661) ANNAM 2 “f ‘SATMONY 

‘ypO OW “MA ‘uorstatg Aqtpend 
pue spoujeyy ‘SoNstye}S [eUONeN JO} SOYJO ‘soymNsuy feonsneis 
euoneN Jo soonoelg uonewnsg puclL (L661) ‘f ‘SA TMONY 


0002 aiquiesap ‘aj}enbua,p senbiuyoe | 


OP-LE “PI ‘sousunig jo1oyfo fo 
joudnor ‘so{dures Sutjos pue suoneuea aun/soeds (8661) “1 ‘HSIN 

“Tel ‘Spl ‘V Gatsog joousuvrg jodoy ays fo 

[OUANOL “S2LI9S QUIN [e1D0s pue ITWIOUOIe Jo JUSUI}sN{[pe JeUOSRas 
Pure UOHeUITss puay [BOOT (7861) ‘f ‘NI@UN 19 “d'd ‘ANNAY 

“LUC-LIZ ‘61 ‘asanbua, p sanbiuysay ‘uotsuauiip suieuyenb oun,p 
inofe :joued sed saygnbug (£661) ‘A’ ‘OULIO 39 “D ‘NOLTIVY 

‘OPS-TES “08 ‘uouDIoossy joousiDIS 

uodidaup ay fo jousnor ‘sainpoooid juounsn{pe euoseos 
pue sa[qeura Ul JoUg (C861) “MW ‘NOSLVM 1° “Vf ‘NVWSNVH 

"Sp-EP ‘LI ‘vo1daup fo Ka1s0g o1aonjop ay) fo suoyavsuvs], 
‘sodvioae poysn{pe Aq uorenpess Uo a}0N °(9161) “A ‘NOSYAGNAH 

“STI ‘sonsneis ul sojou 

91nqd9| JosuudS "61 7-S0Z ‘CSPa ‘He[quosoy "JA 19 UOSUTGOY ‘d) 

‘upuuvopy (y fo Kuowapy ul Siskjoup sataag aully “etiaytIo 

SUOISIAOI UINUIIUTU pue ssouyjOOUS ‘A}[apYy BuIsN soy pus 
a8eJoae-3UIAOU Jo USISEq (9661) ‘d ‘NOSWOHL ? “Vv ‘AVUD 

“pTE-10E “epeueD onbnsnerg ‘sayouipnyjisu0j 

syjganbua sap astjoun ja uoydaou0d ‘76 wnisoducs 

yiendey “N-"g sep ,Asaing uonendog jueuND,, ajgnbua,| 

R uoNeordde daAe sopeulpnyisuo] soygnbus sap inod sinayeurnsq 
(7661) “ST ‘HANVSNVA 2} “V ‘WVdV “WM ‘ada TInd 

“LLI-LTI ‘91 “SOUSIIDIG o1MOUDIT 

pup ssauisng fo jousnor ‘weis0id juousn{[pe [euosees WIV 

ZI-X 94} JO spoyjour pue sontiqedes MON (8661) ‘DW ‘HONd 
9 “WM “TTA “OW ‘OLLO “O'd “TIASNOW “sd ‘AA TONIA 

“IRS-ILS ‘aouadafuog yoavasay jonuuy 

snsuag ay) fo noaing ay) fo ssu1paaz0dg ‘soyei yuawiAo;dwioun 

pue juowAo[duie 93e}s JO UOTeUITsa Joy ASoToUYIa) ueIsokeg 
pure sjopour juouodwioD (£661) ‘S-f “ONVMH 1° ‘d'V ‘UALSdNAG 

“LYE-67E “ZI ‘sousuvig pors{O fo jousnor 

"SIOYIJ Ieoul] [ [XK snsuad pue WIWITAVII-X oui jo somrodoid 
pue uoneatied (9661) “MI ‘AIHO 19 “N “€VHHD “dd ‘NNOVd 

"eMENO 

‘epeued onbysneig yonunpy Ss, 4as—) puv suo1jppuno. — spoysayy 
juauysnipy jouosvay 88/YWINVII-X 24L (8861) ‘@a ‘WNOVG 

‘06-EL ‘II ‘anbusiojs ap auuaipouvd anaas 

DT ‘poyjoul VWINTAV I 1I-X ou} JO Sioq yy Jeouly [euoseas Sunseso1o} 
pue jusInoUO0D ay) Jo sonsodoid jenoads “(€861) Ga ‘NNOVA 


“EMENO 

‘epeued onbysyeig ‘onsojejeo ne qygs-Z] OIOWINN ‘poylaw 
luaujsnipp jouosvag WATXVII-X 241 (0861) ‘aa ‘NNOVG 

“L8S-18S ‘IL “UouDlo0ssp [DISD] UDIMaUy 

ay] fo JoUANoLr “WeIBOId | [-K OU} JO} JOPOU & :solas OUT] [eUOSLOS 
Jo uontsoduioseq (9161) ‘O'D ‘OVILL 3° “d'M ‘GNVTAARTO 

‘TSS-I HS ‘08 

‘UOlIDIDOSSP JOINSIDIG uvI1daULPp ay] fo JOUANOL “SoLIOS pajsn{pe 
Ajqeuoseas Jo uonenseD ($861) ‘AM ‘SITTVM 89 “d ‘aDarand 

‘LIZ-L81 ‘(9 “ A‘ ‘SIOUSTTQN, BUSING JoIANST_ ‘WepsoIsuy 

‘Cspgq ‘ory “YD 19 YRIeYsITY “Y’d) ‘soustmig fo yooqpuoy 
“oun ur Buydues (8861) WW ‘NOTOOUIGIH 9 “Vd “YAGNId 

“S9T-SET ‘SCG ‘SA14jaMouorT 

fo jousnor ‘eyep uoNduINsUOD Jo JOIARYOq SOLES SUIT) BY} UO JOLIa 
Burjdures Jo 1ayJo UL “(€661) ‘AA’ ‘XOOTIM 19 “WM “TTEE 


‘CE-EL ‘Sz ‘alanbua, p sanbiuyoay *| |-x uonestjeuuostesap 
B] Inod saoueLeA sap SIOA (6661) W ‘UANVUN 19 “WM “TIA 
"L@Z-S0@ ‘91 ‘asenbua, p 
sanbiuysaJ, sanbisojouoryp satigs ap ueXoul np sgigdei saSessed & 
sajgnbue se] suep uonewnsy (0661) 'S ‘YAWTHH 19 “UM “TTA 
“eLaquey ‘ansojeyeo ne Q'ZSE] OIgUINU ‘soNsNeIS Jo neoIng 
uelyensny ‘1/66 ‘ON ‘sousneisg porjddy pue somjewioucog suep 
[eaves] op JUoUINDOG ‘sawozjn¢ Aaaing uo uoljOUNISY ajisoduwojD 
pup sudajjog uoynjoy ajdung fo jovduy ay], (6661) ‘Vid “TTAG 
“eLioques ‘ongoyejeo ne Q' [SE] oJguINU 
‘sonsneig JO nesing uelyensny ‘7/36 ‘sonsneig porddy pue 
SOLJOWOUOD SULP [TEAR] ap JUSUINDOG ‘sajnusy 2940] ANOGUT 
uo Bulmaiasajuy auoydajay fo sjoaffq ay) aanspayy 0} uoYoULIST 
aysoduoy puv sjapoyw aovdy aiwig Bus) (8661) ‘W'd “TIAd 
“eliIoqued 
‘ansoje[eo ne Q'gpE] OJOWINU ‘sonsNeIS Jo neaiIng ueleNsny 
‘MAIAdAAQ) UP ‘,Spuas,, BULAOJIUOPW — Satday aul Buijadsajuy 
o1 apind VY ‘(€661) SOLLSLLVLS AO NVAYNA NVITVULSNV 


“eLIoquec ‘ansorejeo ne 9'6979 O9UINU ‘soNSIeIS Jo 
neoing uRleNsny usisag ajduvg daaing ao40.4 anoqvT :4advg 


uolpuoful (7661) SOLLSILVLS JO NVAUNE NVITVULSNV 


"eLIsques ‘angoyejeo ne Q'9[ E] OJguINU ‘sonsNeIS Jo nesing 
uelensny °,,puary,, fo sajpwisy — satuag aul Suiyjoous 
o1 apn Y (L861) SOLLSLLV.LS 4O NVAYNE NVITVALSNV 


aIHdVaIOITala 


“epeurd 
onbysneig ap ‘qeyyD euLION 19 ‘SqV.I OP [lod ANT 30 
dJIJOINS MoIpUY ‘da JJOSD onb sure ‘sasteJUSWIWUOD sind] 
Op SINaJeUIUIe Xe So] 30 JULO[pe IND}OePal 9] SUOTOIOUIOI SNON 
“SOUISIURSIO SID BP Sa[[9O JUOWAIIeSsaogu sed JUs}UasgIdoI 
au gore juasgid a] suep soaumudxe suotuido say] ‘(Sgp) 
soysynjs {O noaing uvypasny | 2 [lounoD yYoapasay 
uD1IjDASNY | Jed sgoueuly 919 JUO xneAvI) sjuasgid soy] 


SINGING YaINGa 


‘JUDW9T[oANOUAI 
ap sueyd xnev yuenb safqeiedwiod suolsn[ouod sap yueIT} 19 
ILI9S B] IP SPIWI9IIXS XN SUOTJLUINSS S90 IP SUON}SOLION sap 
gouryodult |, auraou09 Inb 99 ua JUSWIOT[eANOUa op sur|d 
SIOAIP JUANTeAD (ONT) Ure’ JO 19 [991$ “oseuUOT[NULYDE ,_p 
uvjd a] zed gouonyjut iso mnb Inojovj ‘sogsi~euuOstesap 
SUOTILUINSA Sap 19 BOULPUS} P| OP SUONLUIIISS Sap adeuUOTIN 
-UBYO9 ,P DOULLIVA LT ISO 1D VSTTIN UONENIeAY,P dIQILIO aT 
"s9]U9091 sntd Sa] STOW xIS ap NO SION) ap sopotiad say Ins 
OURPUd} LE] Sp SUOIVUINSS Sop UONRUA ke] op SUOTIeUIT}SE 
sg] Inod siey[Nsgi suog ap ouUOp JUSWE|[aAnoua op ueyd ap 
Q1u93 99 onb Jasuad v Issne JUOUUOP S}eI[NSZI S9_D “STOW XIS 
INS NO SION INS SagsI[BUUOSTeSEP SUONLUINSS ,P SUOTELILA So] 
JOUTWULX9 B 9}SISUOD 9OULPUS} P| 9p UOTJENTeAY,| IS USIg JUSU 
-USTAUOD (Wd) [-Z- | JUSWET[aAnousl ap sueyd say anb anuou 


eHeuuol|UeYOS,P BOULLILA Bj INS JUBWA||SANOUSI OP SUL|A SJBAIP OP JOY9,] -|29}S Je USIE TOW v6L 


(6661) UsrIeOIT “stow sonbjenb op ofearojur eB sogst[eu 
-UOSIRSAP SUONLUNISA P UONLLILA PR] Op USWILXE | OpUuRLOp 
DIQIUIJOP 97199 op UONeN[eAg,| ‘BoUepUd) LT Op sUONeWINSS 
Sd] JUOUTAT[OTOIJJO sed juastNN U saysATeUR Sd] IS SWIQ| 
“smuoIdul09 uog UN sd (Q)Z-7-Z JUSWAT[OANOUAL op ue] 9] 
anb Issne osnl [Tj *sowlia) €] INS HI 2UN,p 39 [|X pou} 
-QUI B] Op JURAIOS 9s Ud SONUd}qO JOURpPUd} FE] Op SUOTRUINSO 
SOP 19 ‘S89BILIOD UOU ‘Sd[PUIZIIO SUOTJBUIT}SO Sap o[JoNsUsU 
UONCLIVA PI OP 39 NCSATU Np sseUUOT[HURYDE,.p DOULLILA 
RB] ANS S}OUNSIP JUSWITTaANOUSI op sued aNenb ap 39JJ9,| 
ISSNV SIQPISUOD (6661) [19 ‘BoURpUd} B][ Op SUONPUISO Sop 
19 SOOSI[PULOSTeSEP SUOTJLUITISS SOP S[[ONSUSU UONLIVA P| 
}9 NVSAIU JJ SIOJ PT B JOIpNIS p JueoOduI asnf uo | Is sqeu 
-UOSIeI STUIOIdUIOD UN 389 (8)Z-Z-Z JUSWI9TJaANOUSI op uel 
3] onb ayuour Z 32 | SOINSIJ sop UoWeXa ‘TJ ‘soULpUd) P| Op 
SUOTJCUITISO SOP 39 SOIST[BUUOSTeSEP SUOTRUIT}SS SOP SANLIOI 
souryodult | ap aj3duio09 319} ynej [I ‘ag39del sJgnbus sun ,p 
uoTTMULYSE | op JUSWAT[aANOUdI op ued Np UOTRIOQRII,| 
Op SIO] UoNbgsuod Jeg “NedAIU ap sogsl[euUOSTesep SUOTIeU 
-1]89 Sap oseuUOT[NUeYIE Pp ddURLIVA kT Inod s}e}[Ns9I suo 


op Issne juouUOp (Ww) [-Z-|[ JUSWE][oANOUSI op sur{d soy 
‘SAN[IJ Op SUOSTLUTQUIOD op aWIWIeS oun Inod souepUd} P| Op 
SOATINIASUOD SUONRUT}SS XNop UD 19g | Op 19 DULPUd} P| 
oP NVIAIU Np SUOTILUIT}SS Sop DDULLURA PT DINPel B BYoIOYO 
UO | IS S}e][NSOI SUOG Op JUOUUOp (MW) [-Z-] JWOUTSTJOANOUSI 
op suryd say “Wuouwgstogid snj[gq ‘SJUSIZJJIP Zosse USI] 
-[eAnoual op suvyd sop Jesi[NN .p spqeigyoid 4so [I ‘souepus} 
PR] JOUTULIDJap & JUSUETedIOULId yUasIA aJQgnbua.p saguUOp 
sop uonesi{yn ] is ‘anb sosued e yUOUUOp IdI snudjqo 
syey[Nsgl say “souRpus) ek] op UONeWINsd ouUN jUdsINpold 
Inb sony sop sJasityn ‘ejao nod ‘ned uC ‘s}us09I STOW 
SOP SINOD NV UONCLILA OP 9GINOD P| JOIPN|E,.p aTessaogu 3S 
[J ‘SowUeINOD sadURpUd} So] JonTeAg,p e[qQeIJ UsAOW UN sed 
1S9,U UOVLIVA 9}}90 Op USWeXd | ‘JUDANOS SQqj ‘sNnOU UOTIS 
"SO9SI[PUUOSILSEP SUOTLUINISA SOp S[[ONSUSW UONLIVA PR] 4S9 
osdyeur | ood ajjenbey] ins afediouud anbysneis e] Is soyqeu 
-UOSTRI JUS *()p-8-p 19 9-4nod-sunp ‘g-4nod-supp anb sj{2} 
‘gTjonjoe indy | & Sst[NN JUSWaT[oANOUaI ap sur|d sa] 


NOISSNOSIG °9 


(9) I-T-T=£ “(8)1-Z-1=H “(S)1-2-1=d 


(8)Z-7-Z=D ‘(p)Z-O1-Z= (8) h-8-p =V NO .,09dNd90 ouUOsiad,, afqeueA eT Inod (VIATAV IT X) p UOsteutquiod e| nod 
ISIOYD JUSUII|[9ANOUaI op ud 9] Inod seus os EP] Op DOURLIRA ET] E OBSCUUOT[NUeYI,.p soUeLIeA Be] op Woddey °7 aansIy 


SO][ONSUSU SUOTIPLIVA :QoURpUdT (PZ 


(w) apnbua | suep slow ap arquuon, 


Stao'c 


0200°C 


$200'C 


0£00°C 


Seo0'C 


SO[[ONSUSU SUOTIVULA :UOTIeSITeUUOSIeSOq (GZ 
(uy) aynbua | suep slow ap arquuony 


of Se 02 St Ol G 0 


® 
' 
' 

! 

1 

\ 

' 

' 

‘ 

' 


e--- 


€6l 


NVAIU Np SUOTeUIT}SA :soOURpUdT, (97 


(w) aynbua | suep stow ap alquon, 


06 Se 02 SI Ol Ss 0 


SOUBUIRA 


NIAIU NP SUOTVUINSS :UONRSITeUuUOSTesaq (eZ 


(Ww) aynbua | suep sioul ap arquuon 
oe Se 0g SI OF ¢ QO 


320 
8Z'0 
o8'0 > 


zeo0 8 
8 


P80 


98°C 


. od 
eo0er” 


«6, 0-8" 
ooo eco oe ve? 


} | | } Las 


0002 aiquiesap ‘ajeanbua,p senbiuyoo | 


‘TU -sUINeAOY Ne VseUIYD 9] 39 1o;dura J Inod (9661) [9018 
Jed g[noTed UONLISLI0S ap a[Qpou Np JURAIaS 9s US SUOISNIO 
-UOD SOUIQUI XNB JUDSALLIL (/66]) [90I$ 19 UsIe' PO “Sowa 
-X9 SI[QPOUI XNop s9d aUS sUIduI0d UONR[sI0D ap sajap 
-OU! op sue eT B Safqeor[dde jus suOTsnoUOd sey] ‘snou 
UO[IS “SOUSIIJJIP JUSUIS[QeUUOSTIeI SUOTIeI9LION sap Ins 
S9SLq UOIL[9LIOS Op So[gpoul xnop saz Inod juswWaTTaAnoual 
ap sueid sIoAIp op 39}J9,[ & JUeNb soleIguss suoIsn[oUOD 
SOUIQUI XNV SUOALLIR SNON “SUOTIEWINSS S99 ap JURAIAS 9S Ud 
sg[hojed sed juos ou sgIpMe souLpUd) EI Op San sa] ‘aseu 
-UO][NULYSY Pp INdLIO oUN .p S9gydejUd JUOS ‘aJgnbua,p suo) 
-PUINSS Sap Ins sagpuoj jUOS soT[o,nbsind ‘nb Wqy4q.] ep 
UONL[PMO9 9p SUOTBUITISO So] INS JUSPUO] as syeI[NsaI sa_ 
‘JUOWUDATIDEdSal “p 19 ¢ ‘Z SUOSTeUTqUIOD 
SI JURSSISIOYD US %p9 19 %CZ ‘%Tp op sules sop yesmnpoid 
(8)I-Z-[ @ ued ne (g)p-g-p ueyd np sessed ‘tojdure,p aq 
-eURA PR] Inod ‘ajdurexe Jeg ‘sa[quigpIsuod suOTwIOI[aUIe Sep 
wesnpold (uw) [-Z-| JUOUST[aANoUAI op sueyd say sed ajjamioe 
OINSY ,| B S9ST[N JUSWIATJOANOUaI op sueyd sap yuoUIIOR|dUISI 
Y] ‘()[-Z-| ued ne inauadns jyuows1es9] 1S9 (9) I-I-] 
ued 9] ‘¢ 19 Z SUOSTEUIQUIOD Sa] INO ‘(W) [-Z-[ JUSWIST[OA 
-noual op suejd say anb suogq Issne juUOs UeLIOdUII JansudUI 
JUSWOYONADYD IdAv JUSUII][9ANOUAI op sueyd say “¢ uOsTeU 
-IqUiOd eB] INog ‘SaN[IJ ap suosieulquios aNenb soz inod 
juepusdepul uoy[NueYyos,| op xneo e sinotiedns sjej[nso1 
sop juouuop inb ‘(g)[-Z-] 39 (S)[-Z-[ suefd soy nesanou 
OP IOS JUDSUAT[AANOUAL op suL]d sIna[[IaW say ‘souRpUd} PT 
OP I[ONSUSU UOTJVLILA Ll IP SUOTJLUIT}SO Sap sed d] sue 
"p 12 Z SUOSIBUIQUIOD sa] Inod juepusdepul 
uo][HUeYdE UN.Nb s}eI[Ns91 sUOG SUTOW op JUOUUOP sTeUl 
‘Sany[IJ Op UOsIeuTquIO. onbeyo inod sanne sa] aonb syei[nso1 
SINI[IOUl Op JoUUOP op JUSNUTUOD (8)[-Z-[ 1° (S)I-Z-I 
JUSUIS|[2PANOUsI op suejd sa] ‘ssouvUeA sap pueIs snjd 
yoddei un neaanou ap juasinpoid uo][nueyse ,| op juewodurt 
JUSUISYONCADYS IIAP JUSUIIT[IANOUSI Op suL{d say ‘souRp 
-U9} BP] OP NVIAIU NP SUOTeUIT}Se Sa] JUIBDUOD INb 90 UY 
agstTyN WWTYWILX/I 1X uosteurquios e] os onb 
gyjenb ‘jnu no s[qrey Jaensusul yUsWIdYONeADYO IoAR SUL 
xne sinousdns JuaInewWop JUeWOdUII [ansUsU JUOWOYONeA 
-OYd DdAP JUSUIT[AANOUAI op sueld say] “(xnes[qe} Soy] snoy 
SUP f 19 € SOUUOTOD) I][ONSUSU UOTLLIVA PI Sp SogsI[eUUOS 
-I@S9p SUOTILUINISS SOP SLD 9] SULP SUITUTW 3So SUOSTRUTQUIOD 
enenb say Inod sgjnojeo syoddel sot anus wed9,T 
"SOULIO) 6 INS HININ B®] ep uoNesiNN,| e puodsal 
-I09 inb ¢ uosteuIquiod ey Inod juawalat{noed “uey0d 
-UII [ONSUSUT JUSLSYONeASYO Sade JUaWAT[aANOUAI op sued 
Say Inod ay10J snd jso souRLIvA BT “NeSATU Np SsagsITeuUuOS 
-Iesgp suoRUINSa sa] Inod ya,dur09 JusWaT[aANOuA e sueyd 
sa] onb suog issne anbsaid sjei[nsei sop JUsUUOp JensusUT 
JUSWOYONVADYD VIqreJ Dav JUSWIaT[eANOUaI op suLd soy 
aonb juanuOUI 9 39 ¢ ‘p xnea[qe) Sop Z 19 [ SOUUOTOD Sa] 
- agdno90 auuosied,, 
Q[QeURA LI ap Sed aI SURP p UOSTeUIqUIOD ze] Inod syeI[NseI 
So] yuaquasgid (p)z ke (¥)Z soINBIy say “uowAndodsa1 


‘9 10 ¢ ‘p xnesiqei xne sgjuasgid juos p 10 € ‘Z 
SON]IJ OP SUOSTeUTQUIOD sap UONeoTTdde | ap sie1[NseI sa] 


worRepodesxa ddA 
SJUVINOIUOD apBdIsEd Ud S213 — VIAINVITX 7S 


‘aT[enjoe sinay_] & astpN USUI] 
-[eAnoual op suejd say] onb sfqrej snyd yuouroou a8euuo][N 
-URYO9 Pp BOULIIVA 9UN JUSINpOId (uw) [-Z-] JUSWaT[aANOUdI 
ap sueyd soy ‘sourpus) e] op JUSWAANOUL JaTUJap np 39 NeOATU 
np suoneunss soj nod ‘anb jyuanHuOW s}eI[NSEI sa] 
(€661) esHPoINg sed ogsodoud [TX op 
uonewrxoldde | op juRAtas 9s ud satqeieduiod syeynsai sap 
NUS1GO 1U0 (166) [901S 19 USIe JOY] “SIOW SION ap sogoedsa 
SUONLWINSS So] 91JUS SOATIISOd SUOTI[SLION Sop JuasInpold 
(uw) [-Z-| JUSWIS][2ANOUAI ap sued say anb 39 stow sanbjanb 
OP S99TBIEP SOgST[BUUOSIeSap SALIQS Sop aNUS Uedg | ‘SI]TeOI 
ud “UoJUSSsIdel DOULPUd} F] OP a[[ENSUSW UONeIIeA eT 
ap SUONeWINsSs sot onb j1eJ Ne JUST) 3eI[NSEI 9D ‘stow anbeyo 
yo[dwi0d jUSWISTJaANOUAI UN B JURpgooId US snUd}qo xned 
B SInoLadns JUSWI9}}9U $}e}[NS9I sap yUoUUOp Inb ‘(g) [-Z-] 
19 (S)[-Z-[ sued say iOS siIno]]IoW sa] JWOUeT[aANOUAI 
op surid say ‘souepua} e] op ajjensuour uOTyeLIeA eI ap 
SUOTJeUINSS Sot Inod 4sa snqd Ind “()Z-O1-Z NO Z-“nod-suvp 
uejd af aonb ajqiey snjd souewea oun yinpoid (g)7-7-Z 
op uryd 9J ‘afduiaxe Jeg ‘JUSWa{e39 9[Q1 UN onof UOISN{SUI ,p 
urd 3] ‘[anuassa InajoeJ UN dINOWap Jansusw JUSUTEYO 
-NBAIYO op gisap a] onb uatg ‘asewIQyd JJ Inod %¢EpP 9p 19 
1ojdura | Inod %¢¢ op sduUepUd) PI Op I[[ONSUSU UOTRLIVA FP] 
OP SUOTJBLUTISS SOP DOURLIVA FL] WeIINps! (8)p-8-y OP Nal] ne 
(8)[-Z-] ued un .p xtoyp 9 ‘ajduiaxe Jeg “USWIa[qQeIgpIsuOD 
SOIpowe 319 DUOP WeLNod ayjanjoe sInsy J B ast{yN ueyd 9] 
39 UOG SUIOW 9] 29 B[quias ¢-1nod-suvp ueyd ZI ‘saIgpIsuod 
JUDUIST[aANoua op suvyd saz sno} oq “yp op ed e oJUOU 
-3ne w puenb yuowapridel soz} onurwip 39 ¢ & | op assed wu 
puenb juowapides say ajUoWISNe SOULLIRA P| ‘soURpUd) P| Op 
Q[JPNSUSW UONLLICA F] Op SUOTeUIT)Sa soy Inod ‘onb juanUOUT 
€ Nes{qQeI Np g 32 /, SAUUO[OD sa] onb Isure ‘(p)|[ ans ey] 
‘JOJJO UIBLI9NN UN 9dJOX9 UOT[HURYI9 | suep 
SONUN SIP SBAISSSDONS SUOISNOU! KXNOP SUS S[[BAIOIUT | 
‘TANSUDUL JUSUTSYONVADY SUBS JUUTS][SANOUaI op sued ap 
seo 9] Suep ‘anb Jasuad e ouuop (g) 1-7-1 19 (S)[-Z-] sueid 
sop onb (9)[-[-]| uefd np spuvid snjd soueuea ey ‘sog[9l 
-109 JUSWISANISOd suOTRAIASgO .P a[[99 onb sajuepusdsput 
suONeAIISgO.p UUdAOU eI Jo[NITeo op sjqeigjoid 3s9 
[I ‘o[Iqour suueAou oun Inod ‘anb yey ne ynoyNs Just) UOT) 
-ENIUS 9NOD ‘[ONSUSUT JUSSYONVADY SOAv JUSWI|[9ANOUaI 
ap sueyd sot aonb sjei[ns9i sino[[loWl JUSUS}}eU ap Ja STOW 
anbeyo juepuadgput uolHueyse un WeuUONdeI9s UO | Is onb 
sye][NsaI suOg Issnev .p jUSUUOP (8) 1-7-1 19 (¢)[-Z-] JUSS] 
-[aAnoual ap sueyd sa] ¢ B | ap assed w puenb juowepides 
QUSUISNe DOURLIVA ke] ‘W-nOd-supp \USUIE][9ANOUSI ap 
sued sap seo 9] sueq *(¢ Nee{qe) Np g 39 ¢ SaUUO[OD say aonb 
Isure ‘(9) | QINBIy eB] JIOA) SfaNsUSU SUOT[MUBYOE Sap JUaLUAYS 
-neasyo ne JusW[ai[esJed syuowIsne souRLIRA Pe] ‘aouRpus} 
RB] Op NVaAIU Np sUOTeWISa sy] 9UIaDUOD INb 80 Uy 


aBeuuoyUeYDE,p BOULIEA B| INS JUBWA||aANOUAI Ep SUR|d SIOAIP OP JOYS,] ‘|29}S J USE P| c6L 


‘O)I-I-1 =£ “(8)1-@-1 =H “G)I-7-1=d 


(8)Z-@-7 =O “(p)Z-O1-Z =  “(8)P-8-F = V 20 29dn990 auUOssod ofquieA BT INod ([ |X) | UOSTeUrquiod eB] Inod 
IStoy JUSUIOT[aANouI op uezd 9] Inod afeuIsUO aLIgs L] op DouLTRA P| B BVUUOT[HULYIO,p soUeLIA e] op Woddey *T aansig 


So][ONSUSU SUOTeLIeA :gouLpUdT (PT 


(w) apnbus | suep stow ap arquuony 


0€ SZ 0¢ St Ot S 0 


$c0'0 


il a C9056 


ng 0€0'0 
$£0'0 
0r0'0 
$70'0 


0s9'0 


So][ONSUSU SUOTIELIVA :UONesITeUuUOSIesaq (GI 


) (w) azpnbup J suep stows ap alquony 


Of 290-0645 G6 P60 00550 
$ tg, 
8 


“SUOT[HULYIE Sap JOENSUU JUOUIOYONeADYO op sIZep sUIgUI 
3] JUUasgid Inb JUsWaT[aANOUAI ap sued sap Inod surgUI 
Rl JSO B[JONSUSU UONLLIVA L] Op Jo NVOATU NP suONeUIT}sS 
SOP DOULLIVA PB] ‘agsTTeUUOSTesap ales P| Inod ‘anb JuanuUOUL 
(q)[ 32 (e) | sonst say ‘onbryeid us 1a}dope | jnoed ou uo ‘Zz 
UOTIOIS BP] B UUONUSUL B | UO SWIWIOD ‘sTeU ‘JUSWATTOANOUSI 
op sed & K.U [I NO aI[99 Isa UOIINIOS sINaT[IowW eT “s}eI[NseI 
Suog Op sUUOP JUSWIST[aANOUSI ap sue|d sao ap uOHESTTNN | 
‘alensny ud jo epeurD ny ‘jUsWOaYyoNeAsyo op sed ev AU 
IE oyJenbey, inod uoj[Hueyoe | ap uonsodoid ey a11p-e-3$9,9 
‘m/| OP UOOUOJ oUN ‘yIeJ Ud ‘jSa w-snod-suvp \UdWII] 
-[@ANoUdl op suejd xne agIoosse soueLeA eT (¢ nes[qr} 
NP p 19 € SoUUOOD say anb Isute ‘(q)] sINBIy ey JOA) aysaytU 
PUL 1S9 JURLOdUIT JONSUSUL JUSUTBYONeADYO UN Pp VBeIURAL | 
‘Q][ONSUSU UOTIVLIVA LI Vp SogST[BUUOSIesap SUOTRUISA Sap 
seo 9] suep ‘Juepuaday ‘aqeiogidde uorjeloljauie ounone 
ywnpoid ou JonuUe JUSWIOYONRADYI Jae JUDWIT[OANOUSL 


L6L 


NVIAIU NP SUOTVUITSS :soURpUaT, (I] 


(w) aynbus | suep slow ap alquuony 


Ge Sc 0g St Ol S Q 


SURE A 


0200-40-00 040 0060 seeovorered 
| i | | | 


NVIAIU NP SUOTILUINSE :UOTeSITeUUOSIeSaq (P| 
(w) aynbua | suep stows ap alquon, 
O& Se 4 Sk Ol g Q 


SOUBLIEA 


24 00095-0966 6945. : c0'h 
OO 9.9.0-0 Wy g 


4a $0'L 
| | | | | 


op sued op uonesi[yn J ‘Nesalu np sogsijeuuOsIesep suoN 
-RUINSS SAP JOURLIVA PB] DUIBDUOD INb 99 UD s}eI[NS9I SUOG 
Op JUSUUOP [ONSUdUL JUSLUSYONVADYO SUS JUSUAT[OANOUSI Op 
sueyd say aonb yuaUOU! ¢ NeITqQe} Np Z 12 [ SOUUOTOS SIT 
‘JUSUOT[OANOUAI op sed eB 
A.u[t,nb atjtusis Q¢-1nod-supp yusulay[eAnoual op ueyd up, 
‘O€ B | Op aUvA WW NO W-4Anod-suDp JUSWAT[2ANOUAI op uel 
9] Inod 3a (w)p-q-p sueyd suey Inod onbrydeis ua sgu0d 
uos dao duro sjqeueaA Pe] Inod sjei[NsoJ sa] “asn[OUl Js 99U 
-uOTjoaTas 9yIUN ouN Nb stoj ap [e}0} SIQUOU Np UONOUOJ US 
NBIAIU NP J[eUIZLIO UONeWINS | op soUeLIVA eT Jed SgsIAIpP 
QLI9S LP] Op Uj P| B DOURPUD) P| OP SUOTLUIT]SO Sap 39 SogsTTeU 
-UOSTESEP SUOTJBUIISS SOP I[[ONSUSU UONCLILA kL] Op Jo NIA 
-IU NP sseUUOT[WULYSE Pp SOULLILA L] JUDTUOU (P)T & (2) 
SoINnsIj sa] “¢ neayqe} ne sayuasaid yUOs (| UOSTeUIquIOD) 
LIX sodA} sony say quesi[yn us snuajgo sjei[Nsol so] 


$}U¥1NOIUOD Sad} apEISED Ud SAINI — TIX I'S 


000zZ eiqwas—ap ‘ajenbue,p sanbiuyoo |; 


ee Uh ee OST EV ORT OL TIT LIE 


op sed 
7OT ELT 961 STZ 60 ZEO 80T IIT g-4nod-supp 
OL'T O6T ¥8'T 00% Z7S‘O SEO LOT 60°T 9-4nod-supp 
891 LET Pst 007 ISO SEO 90'I 80'I (Z1)9-9-9 
691 Z6T €9T ELT 950 Ih0 OI SOT (8)p-8-7 
ET PET PST 9T 190 8r0 SOT LOT (Q)E-€-€ 
ECan GVA Coen OSaT'a mt OL.0) 109 0) = LO; 00r1 (p)Z-O1-Z 
601 860 Lrl LSI IL0 090 ¥0'T 90°I (8)2-Z-7 
1I0T 260 6€1 6YI ODT OOT POT 90'T (9)I-I-1 
r8'0 OL0 IZI 871 660 L60 ZO'l ZO'l (8) 1--1 
LXONSLOMGTT SGI 00 1) 660) 201) COT (S) 1-2-1 
OO'T OO'T OO'T OOT ODOT OOT OOT OOT yajduroo 


“wIgYyd “990 =UIOYD ‘990 WOYdS 990 WOYd “990 JUDUWIS][SANOUSI 
us i WS Aa VS 2s VS Spd 
(p UosteuIquioD) 
juepuadgpul sseuuo][Hueyos .p ueyd un 
Inod aseuUo][HUeYo9 ,_p OULLILA FE] B JUDWIO[[9ANOUaI ap suvyd 
sureyso Inod a8euUO][HueYs9 ,p soUeLIRA eB] op Uoddey 


g neayqey 

‘ ‘ < ‘ ‘ A ; JUOUWI9|[9ANOUSI 
YOON iP yiOunOS: lnc ula ty On SC On eLlslavcol ap sed 
9L'0 190 ZHI 7ST ISO veo PIT LIT -4nod-supp 
180 OL'O OF 6rI ESO 8E0 ELI SIT g-4nod-supp 
C8:0n69, ON OG LunLYs beeeS OwiLS Ow -Clal «FUT (Z1)9-9-9 
€60 S80 SEI Il 850 pO ITT ZIT (8)p-8-1 
COMmCOMEACS Iepltalen 69,0Pe1S Om cl Lb SIL (Q)€-€-€ 
SOMMLOIE OG Ie CEO, 19:0) "9011 SOT (y)Z-O1-Z 
$60 680 6I'T €7I ILO 090 901 LOT (8)Z-7-7 
ZL0 090 80° IT‘l 660 66:0 ZOT 0'I (9)I-I-1 
LL‘0 790 660 860 L60 €60 660 L60 (8)1-Z-1 
6L'0 890 66:0 660 860 960 660 660 (S)I-@1 
OO'T OO'T OOT OO'T ODOT OO'T OOT ODT yo[dur09 


“WIQYD “990 “WQYD "990 “WQYD “990 “WIQYD "990 jyaua] JaANOUSI 


q I+? q I I+! q 


il L VS VS VS 
(¢ uosteuIquio)) 
juepuodapul sseuuo][Hueyse ,p 
ueyd un inod aseuuo][Nueyo9 .p oUeLILA FB] & JUDWII]JOANOUSI 
ap suvjd sureyiso Inod a8euuo][NUeYog .p soueURA P| op Woddey 


ap ueld 


g neaiqeL 
S£05 650 Stil 6805 CVO roo! Tht Vint ap sed 
€60 780 I9T SLI 6r0 ZEO 80 ITT g-4nod-suvp 
IOT 760 9ST 69, 7S‘O0 9EO 801 OTT g-4nod-supp 
IOT €60 951 LOI 70 SEO LOT 801 (Z1)9-9-9 
Slots Clive Lyal 9S) 95:0) 0) 90'T 9011 (8)b-8-r 
Ol 6LT Ol 6rI 190 6r0 90°T 80'I (Q)E-E-€ 
A CLE ell Cree LOE OKO (KOE ray! (y)Z-01-Z 
760 180 971 ZEIT ILO 090 FOI SOT (8)@-7-7 
LLO S90 LUI 71 OOT OOT E01 0! (9)I-I-I 
8L0 990 SOT LOT 660 960 O01 O0T (8) 1-7-1 
080 690 SOT 90T OOT 660 I0T I01 (S)I-C-1 
OO'T OO'T ODOT OO'T OOT OOT OOT ODT yo]dur09 


PAL UU UU UE Ue ee ee 
“WOYD “990 “WOYd *000 “WIOYd ‘990 “WIQYd *990 yuoUIO]JaANOUaI 


giee Us TiS nce Ae i Pla op weld 


(Z uosteurquioD) 
juepusdepul oseuuo][NueYys9 ,.p 
ueyd un nod e8euuo][NueYs9 ,p IOULUIRA FE] B JUSWITJOANOUSI 
ap suejd sureyss nod a8euuoi[nueyoe ,p soueueA ve] op Woddey 


p neaiqeL 


S60 SL'0 69'I O8'T vr'0 v0 sor o'r NONIONN 
Ell 90 SOT SLT OS‘O €€0 80'T 60'I  g-4nod-sunp 
CWT 71 «POT OLT «750 SEO 80'T OIL  9-4nod-sunp 
CGCCL V9 Vs OL CSO 95:0) 80:1 Olsl (Z1)9-9-9 
vel THT LST 69 LSO @rO 80T OL'T (8)p-8- 
STI 671 rT ~SI 190 8r0 90T LOT (9)E-€-€ 
OT CET OFT SET ILO 190 OT HOT (p)Z-O1-Z 
S60 £80 €ZI 97T ILO 190 ZI 701 (8)2-Z-@ 
CSO MOLOML ee lal OO cee OOK aml Oneal Onl (9)I-I-1 
LLO 790 66:0 860 660 L60 660 860 (8) 1-2-1 
6L0 890 O0'l 660 O0T 660 660 660 (S)I-@-1 
OO'T OOT OOT OO'L OOT OO'T ODT OOF yojduroo 


“wOQYyd “990 “WIOYD “990 ‘WIQYDS 990 “uIQYyd “990 JUDWIOTJOANOUSI 
peng a Vss vs. tVS SPA 


({ UosteurquioD) 
juepusdgpul adeuuolnueyse .p 
ueyd un inod aseuuo][Hueyse .p doUeLIeA E] & JUSWOT[aANOUSI 
ap suejd sureyis9 Inod aseuuo][HueYyoe ,_p souvRA ke] op woddey 


€ neaiqeL 


"(666 Wore OJAI) SOTQe[QuUIAs soTeI9UES SUOISNTOUOD 
Sop & JUUDU SLIVS RI Op Nati a] Inod sg[nojeo syioddes 
SY] ‘slow onbeyd uoj[HUeYdsg J JUSWIA}Q;dWI0D jWe[oANOUSI 
uO, | IS WeIpuengo uo] enb sseuuOT[HUeYIE .p VoUeLIVA P| 
@ SOgIPNI SUOIRUINSS Sop eseUUO][HULYDE ,p VOULLICA PT Op 
yoddei 9] “uowaT[oAnoual ap sueyd sureye. nod “uouUOp 
Xnva]qe} SdD ‘apeosed Ud SaI[IJ op UOsIeurquios onbeys 
Inod jusuIa][aAnoual op suvy{d SIAIp ap SjoJjo Soy JUOWINSOI 
9 & € xNeaIqQuI SIT “ALIVs PT Op sIWAN KA 93N0} ke] B oULpUS} 
R[ op SUOTBUIT}SO Sa] aonb Isure sagsifeuUOsTesop SUOTeUINSO 
sg] Inod ‘ued anne ,p ‘ajansueu! UoNeURA P| B 39 ‘Wed ouN.p 
‘neoalu ne jUuapuodsaod.Inb sanylj say suosI[yN snoN 


SLYLTNSAY ‘s 


"agAlasqo apoued eB] op INOjNe oNjOAg 
Q]J991 Olas ev] JUOp UOdSe] ve] op pusdep oseuUOT[NULYDY,p 
INI] & onp ajuesoduod e] op saNvjes soueyodurt | anb 
yuayeysuoo sy °°" im - '%m sed Jayuasgsdas ynod uo J aonb 
‘QOULPUD} BP] OP SA[LNIUL SUONLUINSS Sap S9AIJOSGO UOI}DELI09 
RB] Ins JUSUII][aANOUSI op suL[d SIOAIP Op JoJJO,[ JUeIpNIE 
(Q00Z) U2Ie POW 19 [2201S “aes ke] B SUONeUIT}s9 sop ojnofe 
uo,| enb ainsoul B soylej GoURpUD) P| Op SaleITUI suOTeUL 
-I]S9 SdP SUOTIDALION XNe NP Iso UONENIIS 9139D “UOISIAgId 
op sindLla,p asneo & SeyTWIANXe xne JUSWISLOJ dJUSUISNe 
SUS eB] op o]e}0} souRLeA ek] aonb juaNuOU jo sg}TUN 
buto suroul ne jueyoduos stup-s}e1q xne JonueYo Ud sastul 
S29] INS dLI9s BP] VIPNID JUO (666!) JoWeIY 19 [Jeg “OUE] 
-[eAnoual op ued of suduioo A ‘a8euuoyjnueysg,p ued 
np sed puodeap ou ‘aseuuo][nueysg ._p IneLa sues BgAIOSqo 


eGeuuoyjUeyoE,p BOUEIEA eB] INS JUBWA||eANOUAI Op SUC SIOAIP AP JOYS,7 :|991S 18 USE TOW O6L 


augs ey e puodsazo0o nb ‘ouLia) swigIxnep a] ‘uoTIsod 
-W099P 91190 op sULIO} JorWoId 9] INS JUSNIJUI JUSUWOT[SA 
-noual ap sued sIdAIp JUOp UOSe} eT SIPNI9 SUOAR SNON 


Ge at 474 ee Vl S+] 1 Vl 
( A x - AM) + CAM - AM) = 7, ( eM — Aim 


21s 
QUIUIOD VOUDIOJJIP 91390 Josodusosep suoAnod sno “UOISIA 
-g1d op inoue, pour mb “**’_ 2am —"d%m aouslajjIp ek] 
JUSIEPISUOS (666 ]) JoWeIy 19 Tog “A AM eras osTUIUIOD 
1Iv19,U oseUUOT[NUYE Pp INdLIa oUNONe Is sUOLIPUsTgoO 
snou onb geuly Inojea ey “uowriorg °,’m spriod 
ap onbinguids yeury 1naj990a un inod ‘**’4 7, m arti909,s ined 
ajenbusa ,p suoneuinjsa soy Ins saseq J sdu9) ne seuly con] 
Ingjea eT ‘a[quiogidde uoneuea ounone snjd suojojou ou 
snou ‘s +7 Yurod ureyioo un p inured e stew ‘Iosueyo yeunod 
1 sduro} ne san]lJ Inojea ve] ‘aLIgs e] B SUOTJBUIT}SS sop 
suojnofe snou sumui0d 74 7m - 4 ’m 389 Il sgu9pIsuOd oSeU 
-uo][HueYys9 ,p Inoue J “x ’M JS9 BSTUTWIOD 11e19,U aBeUUOT 
-[QURYD9 ,.P INdLe ouNONe Is SUOLIPUdTIGO snou onb INIyeA BT 
“4/4 sant inajea ve] suuop Inb 90 ‘74 alp-e-1s9,9 ‘7 sdura} 
ne surULIa) os Inb suoneuUtss.p onbisofouosyo stgs ef] 
ap JURAIAS SnoU Ud J Sdura} a] Inod souRpud) J ap 39 sagst[eU 
-uOsTesep suOTJeUITsa sorgmuoid soy suo[noyeo snoN 
‘(Z) SUBP DOULLICA Op SULIO} 9] JULIJIPOW Ud UOTeNIIS 9709 
op ajdwiod s1u9} yenod uC “[ TX Seni sep 19 VWWINV 
d[QpoW Np x1oyo 9] JooueNTJur,.p onbsu soinotgjue sopouied 
OP SIO] SSID oSeUUOT[HULYSE Pp INdLIA | “IO ‘sainaLigue 
a1gnbua ,p saguuop sap saide .p VINTAV 22epoul a] owso 
UO 39 SIJHUSPI UO UOUTAT[ONIIGeH “InoLIa ounone jwusI0d 
-W09 OU VINTAV 22epoul np suoneorioads say aonb osod 
-dns uo ‘ojdurexe Jeg ‘uotyenys ey enbijduroo ‘WIT VILX 
spoyjsul eT op saiped a] suep ‘YVIATYV suorstagid 
op uonesi[yn J ‘onbryeid ua jusuTUTeINOD sgst[yN xnIo op 
_ Sfequesgidas quos tor sgIpmMe WAWTAVI LX Se[epoul so] 
‘JUDWI9T[oOANOUaI op sued sop 19 VINTUVITX 
Op suONeRSI[esl SaJUdIOJJIP 19 DOURPUd} B] OP SoII[IJ Soy 
Issne SUOJOPISUOD SNON ‘aJgnbus ,.p saguUOp sap Ins sapuoj 
(44|44),4 9p uoneumse oun Jasin, p 39 UONesteuUOSTes 
-2P Op spoyjoul e] op sjqeuuostes UoNeUITxoIdde oun I1u9}qo0 
B UOSeJ ap SISIOYD spiod sap dee (Z) UoTeNbe | op yURAIOS 
9S UD SA9SI[PULOSTVSAP SUOTIRUIT]SO SOP SOULLIVA PI JOUII}SA Pp 
gsodoid yuo tnb (1 g6[) Inosuoyy 39 J0}[0 AA Jed agidope ayja0 
anb suigul eB] 1Sd aSeq OP SPOS 99D “slow UN.p pedE 
un inod ’m — '*'m sastjnn ined uo ‘ajduioxa seg “juesueYo 
senbrdde sptod say stew ‘owgui oy ayso1 44 saguuop 
@P INB}90A eg ‘suoNeLeA set oumso uo, Ig “(4A|l My 
ap sogudoidde sinayea sop uonesi{yn,, & 20R13 Jol] 
-nomed juourajaanouss ap uvyd un Inod sseuuoy[HUeYoe ,p 
aouRLeA eI Iuaj}qo Inod (7) uonenbs,| suep s}inponut 
ang 1USANEd $1Nd}99A Sad “soURPUD} LI ap s[vUIy UONRUINSE | 
Inod jounsip sptod op 1n9}99A UN yo UONESTTeUUOSTESEp PI | 
Inod o8enlj ap sptod ap 1najoaa uN yUINOJ yUepUodsaLIOD 
OPLdS¥d Ud OII[IJ 9] ‘SONI op uosreutquios onbeyod nog 
‘onbneid ua ostjn uo | onb sinansuoy sasioatp op HINA 
Sap INS Saxe SoI[IJ op oUTWIeS oUN Inod JUOUIST[aANOUAI Op 


681 


sueyd SIOAIpP Op JoJJo | JouUeXd ,p JUd}JoULIEd snoU seIsTOYo 
soleguiy suoneuitxoidde say] ‘seo sinaisnid e sarqeordde 
qUOS ¢ 39 Z SUOSTRUIQUIOD sa] ‘(€Q6I) WINSeq UOTIS 


“sojueLioge SINe[eA soy Inod 
UONPIIJIPOW suNoNe 3yJOdUIOS ou sTeU 'p( 2148°0- 1) 
(gg0-D='(.7-D(-1) suoy eB ap 
a[gpoul un sade ,_p segfodeynxe suotstagid sop onb tsure 
79 19 "1 SETH sauTly Se] puerdwios anyly 2D 
WWIUV SuoIstAgid 99Ae UOT [TX Bpevdsed Ud OI “p 


“soqueLoge 
SINgTeA Sa] Inod uoNBOYIpow sunone sOdwO9 ou sTeUL 
“D(.,8€0- I(GE0- D="4(2,2- D(T- 1) eunI0 
RI Op s[gpour un saide,p sagjodenxa suorstagid sap aonb 
isure ‘S**z¢ ya ***] O61 saniyy say pucrdutoo any 99 
“WVINTUV SUOISIAgId DaAR UNOS [ [K Bpeosed Ud ONL °*¢ 


"soqUueLIage SINITeA 
Saj Inod uoNeoIpour ounone syodu09 ou stew ‘(oueTqG 
1INIQ) d]JaNpIsal INaLIS,| ap JUSWIS}Te.N ap Ind}oRJ uN ‘’y 
39 uoNe[OdoNeI op InayeJado ,| 1S9 J NO ‘ O(a; g90- 1) 
(av'0- 1) = '4(2,8- 1)(@- 1) euusoy e] Op WWIUV 
gjepou un seide.p sogjodenxa suorstagid sop onb 
isure ‘S“*e 9 49 **€] ote TH sanyly soy pussdwioo anty 
9D : WWI suolstagid sane odA} [|X Opeosed UIST °Z 
“sojuRLoge SINa[eA Sat INod uONedyIpow sunone sy0d 
-wioo ou ste ‘(***7 9) sraruuostes sina}oRJ sap uONRUI 
-nsa,J nod ¢ x ¢ ummoun ‘“(***] ¢) syatuuOstes sna}oRy 
sop uONeUIIse argTWeJd ey] Inod ¢ x ¢ wu uN “(¢][H) 
oUuLpUd} BF] Op UOTWeUIISA | INod souLIa} ¢] INS HW 
oun pusidwios any 99 :0dAi [Xx opeosed ud ONT“ 


‘juoatns nb YWTYVILX oP 39 TTX oP 
SOUIDJUL SANIIJ SAP SUOSTRUIQUIOD Sd] SUOI9PISUOD SNON 
“WAWITAVITX 20 TTX epoyigul 
ey onbijdde uo nb uojas jusJajJIp SogsI[euUOSTesep SUONeUL 
-INso xne Ja aoUepUD) BR] Op suONeWINss xne sonbijdde apes 
-SB9 Ud SAI] So] ‘OLS BP] Op SoWWIONXS xNY ‘soURpUd} eI Op 
any un sed gstpeuuostesep on] lf 9] JUeITdH[ NW Us soURpUd} 
PR] JOWINSS Inod apeosed Ud oIIIIJ 9] SIOTS SUOUD}QO SNON, 


Lo(.a°"*S.q)"'HI°*S.a-1=$ 

ed 
sojuasgidal JUOS opedsed Ud UOTIRSI[VUUOSTeSap Op SANII SY] 
‘d- 1 =, 2dwoxo sed ‘any un p yuourg{du0s 3] onbrpur 
2 UONRIOU LY ‘aINUAPI,p soLyeU oUN ‘] Jed 39 soul} Z] 
Ins 992u90 WU oun q Jed suojuasgidal snou ‘uljUg “oyTeU 
-UOSIES OP SINIOVJ So] JOWINSS Inod saoLyeUI sd suOSTTHN 
SNON °¢ xX ¢ WU Be e puodsaios nb aryao ‘***g sed 190 
€ X € (WU) sJIqoW sUUaAOUI Le] B JUapUOdsea1I09 Inb sprod 
sop someu ey ***g sed suoquosgiday ‘sonbinguiAse nb 
sure sonbiyawiAs sanylj Sa] Inod sou49} ¢ | Ins UOSIOpUudH 
op safiqow souusXoul sap asenily op sptod say yuou 

-UdTUOO sous] Soy UOp somneW eT *'Fy Jed suojuasgiday 
“SOPpOYISW SdIQIUIOp xNop 
$99 op a}stjeo1 uoTeUIxo1dde oun sI1Ud}qQ0 Inod sydo1dde 


000z aiquiasap ‘ayenbua,p senbiuyos | 


Ing] SUOST[NN SNON "WWIAVILX 1° [LX Sepoyjeur sa] suep 
S9SI[IN SSUIULIO}9pod SoMUI] SONI] SIOAIP op UOTNTOAUOD 
R] OP JUI[NseI opedsed UD SaI][IJ So] NO ‘apeosed ua pou} 
-9uI oun gsodoid yuo (9661) NIYD 19 qeyYyD ‘win3eq ‘Uo 
-anbiun saguuop xne snuajgo Isure sptod say sanbiydde sind 
Jo SaN[ sop uonesgpuod e| e VITYV Uoneodenxa | ap 
y9JJ9,| JoIsg\uI ynod uO ‘(japoyy a8n4aap Zuiaoypy paywssajuy 
aaissaisasojnp Inod WIATYYV) 2189IUl safiqow! ssuusfou 
B Jissaisgiojne sjapowl un,p uoneorydde sed sgywenxe 
XNOp XNv I[VUISO OLIgs e] Jajodesyxo e ajstsuod Inb [ [xX 
QPOYIJSU! Ll OP UOISUD}X9 OUN Jso ‘aTTa B JUeNd “(QR6T ‘OR6I 
wuinseq) WWIAVILX spoylsul eT “TTX spoyjsur ve] op 
sollegui] suonewrxoidde sap yinposd quo “(¢66 1) aystoInS 
19 (Z86T) SHIRA “(9L61) OPIL 10 puefeag[D ‘(g961) 3uNOX 
quop ‘sinoyne sInsisn{g ‘| [XK spoujewi ey Jed sayinpoid souep 
-U9} B[ OP SoTLULJ SUOTFRUITISO Sop 39 SdQSTTBUUOSILSEp SoTeUIy 
suonewiss sop uonewtxoidde oun ouuop sosteguly sony 
op USXKOU Ne aBeSsI] FY] “SLIVS Op Ulj Op Jo INGEp op sinaTeA 
S93] Inod sonbinguiAse san[lJ sop 39 sayeNUS. sINajeA so] 
Inod onbinswiks any un yuastnpoid mb saytqour ssuuskour 
Sop JUSWISATILIOI Jonbiydde e ajstsuod | [x epoyjewl eT] 


GONVGNGL VT Ad SNOILVINILSG 
Sad Ld SHASITVNNOSIVSAd SNOILVIILSA 
S4d SHYIVANIT SNOILVIAIXOUddV ‘F 


"Zz negyqe} 
ne soguuop sinejea soy onb yuowopides snjd snuruip mb 
‘,d = (S)y alIp-e-1s9 9 ‘g°Q = d UONeIa09 oun ‘uoNe[ndod 
RI Inod ‘saae ‘aydunts onbiygui0as souessioloep op sep 
-OU UN B JUASST9gO aJgnbus_,p suOneUINsd so] aNUS SUOTLIAI 
-109 so] onb efao nod juesoddns s]J ‘s}ounsip jUSWO][9ANOU 
-d1 op sueyd ap aiquiou Wed un Inod xnij sap 39 xNeaAIU Sop 
QOULPUD} EP] OP SUOTILUIIISS Sap 39 SogSI[BUUOSIesap SUOTILUL 
“hse sap sedAy sinaiie se] WarpMyg (C661) 227 19 AIJHOINS 


400 6800 ©6800 6600 600 O10 10 Ilo (SP 
CHOMME CCU MmROTO MSO UE LOOR YO." TSO) 47910) G4 
8 L 9 S v € (4 I 

sInswIoYyo ap uotodolg 

OOM OM MUnOM schONmelOMePliO™ (SLO MESTO Tp 

960 Or0 SrO0 O0S0 L450 90 IL0 O80 = (S)4 
8 L 9 S v € c 1 e8eeogp 
sagdnod0 souuossad ap uonsodoig 


VdvVd — suoneisuosoiny 
¢ neaqe | 


“sguUUOp Saseeoep 
Sop R[ap-ne singfea say Jajodenxe inod (g661) [lod Jed gsod 
-Old a[Qpour UN Pp IAJaS 1S9,S UG “JUSWa][oAnoual op odnois 
NP NedAIU Ne UOTLI[aLOIOINV | SUIUIOJ9p B UO ja SUOT} 
-edgi sap suIWIOD Wd V4.1 ep JUaUIaT[2Anousl ap sodnoi3 
SO] 91TeN) B UO ‘I1U9}qGO Sey INOg *Z NeaTqe} Nv sagjussgid jUOS 
“(WdVq) 2anoe uone[ndod ey] Ins ouuarjensne ajgnbua | ap 
$3911 SINSUIQYD ap 39 Sagdnd00 souuOsied ap suonodoud say 


OIIP-B-IS9,9 ‘SIQST[N SINI[VA SFT (866) [19g Jed sestteas 
9pNj9 SUN .P Sagi WUOS (¢€) SUP (S)p Ja (S)4 ap SUOTJeUIT}SO 
sa] ‘dnooneagq area ou uoneindod x] tu a8euuoT[HUeYse .p 
uejd 9] tu onb Issne pusjue-snos uonendod ey] op neoa 
-1U Ne UONeI9LIOD eR] Inod ‘(s)p ja (s).4 ‘satquis suOTIe{III0N 
-OyNe .p asgujoddy | JoseVsiaug ‘sanly Sap satoojJo Inonsuoy 
RI Ins suloul np ‘nel] eu uoNe[ndod ve] op aimjonns ey] ap 
Iu sseuUOT[HUeYSE .p ueyd np ajueLOdu UONeoTIpou suNdD 
-ne nb pusjue-snos 9}Ue}SUOd Jso SSeUUOT[NULYDE,p SINdI 
-13 Pp dLI9s BP] op soURLIVA eT] o[[onbe] uoTas oseujodAy J 
"JUDUIS][2ANOUAI op sueyd 
Saijne .p JUaIgdssns onb saguuop op oinjonN}s ke] op JuaINosIp 


(2661) ‘1109 39 JoT[N{ onb tsure ‘(gg6 1) No[Zopry 39 Jepurg 
‘'a 19 ‘vo e agio0sse UOT}V[PUOS LI Iso (Z)p onb spur} 


SS 


‘*p yo ‘y ‘suosip ‘ap jueudaoid UONRIPLIOD RJ 189 (Z)4 ‘19] 


'y 7a €f of if % tj q% p 
ty \y 3 £3 zg 1g f ve € 
£9 to iP) ¢. &p tp \p 7) Zz 
*q «9 tq \g *n fp tp Ip I 


JUSWa]JaANOUSI 


LEP IS ee HS pti CPCs | +e ap adnoip 


goualgjal ap apouisd 


p-4nod-supp yusuIS|[2Anoual op ueyd np osmjonns 
T nesqeLy 


‘sonbipur 
QoUusI9JOI op spouted e] e nbsnf ajgnbus,p uojjHueyse ,| suep 
Snjoul 939 & [oued 9] onb s1oj ap o1quiou 9] onbrpul sorput, | 
JO SQUDIDJJIP Soa] sop Jed soyuasoidos yUOS spoued sIOAIp 
SY] ‘slour yInYy ap sported oun ins gnbijdde p-«nod-suvp 
JUSUIOT[9ANOUAI op ued op sjdwioxe uN oUUOp | Neo[qei JT 
“(LL61) Sous 39 WWI ‘NOdS Jed 9ALIEp Inj90 aonb suIQUI 9] 
JUSWIS][INUISSd ISO S[QPOUI 9D “sed oI Sed 3so_,U BIVD ‘TeIQUd 
ud ‘sIeul ‘9 = (S)p sioye ‘syuepusdgpul UOs JUSWIAT[oANOUSI 
ap Jarjnornsed odnoss un,p amied juoj inb sjaued sisalp soy 
Ig ‘sloul onbeyd uo[HUeYs9 | op spwtddns jo sgynofe yuos 
S$]USUISSZO] Op sIquIOU SUIQUI 3] aonb osoddns uo] Is “UOWON 
-ne [NU 3sa 9 | — Ww‘ “Q = S ‘u/s — | = (S)¥ ‘tu-4nod-supp 
JUSWIaT[eANoual op uvjd un Inod ‘ajdurexe Jeg ‘suoT[HUeYSS 
sop JuoUIayoneAsys op (S)¥ INooRJ 9[ SUTULIOJNP USUI 
-9]]a@Anoual ap ued a] ‘aousIgJal op sopouied xnap saj ous 
UNUWIWOD Ud UOT[HUeYIE | op uoTHOdold eT sJUeseIdal (s)y NO 


(¢) ((s)p - (s)4)(s)¥ + (S)p = (S)y 


yso “*'4 30 'A anuo oseuUOT[HUeYOE ,p 

uoneali0o et enb juapusjue-snos sasguyjodAy sap 
“sayuepuadapul jus “uowereneds sogredgs ualq 39 Souda 
-9IJIP AdN sep & Jowejeniiqey JUspuodseod Inb ‘juew 
-9]]2Anoual op sadnois sJoarp inod sanua}qo SUOTeUINSS So] 
aonb Jo JusWIaTJaAnoual ap odnois onbeys nod suoneurtse 
sop suusXoul B] & ‘JUOWOANeUNIXOIdde sutoUl Np ‘afes9 3sa 7 
sda} ne uoneumsea | onb issne suosoddns snoyy “}UauaT[9A 
-noual no ev A [Is (S)p 32 Nol] no &,U JUOUWT[eAnouUal 


aGeuuojUeYOS,p BOURIEA | INS JUBWA||9ANOUAI BP SULId SJOAIP OP JOYO,] *J8EIS 18 USJE TOW g8i 


unone Is (s)4 480 o][eAIONUI.p sopoued s e JUSUIOT[9ANOU 
-a1 op odnoiz owigui un p sgidne sonuajqo suolewinss so] 
anus sseuUo][Hueyos .p Uoe[a0d ey onb suosoddns snou 
‘juonbgsuod Jed “(0661 ‘T109 39 Y8UIS *7661 SAV HOA) sus 
-IOA SoseusU op [oued ane un Jed goe[dual so aygnbus | 
op gurus iso Inb Joued 3no} Jo sgiZap sinaisnid Be aseUUO] 
-[Hueyoe | Ins juoindde.s YqIWa sep uednyid ey yuows] 
-[eAnoual op odnois un ouLIOj UOSeJ 99399 ap salfoI sjoued 
op gquiesud,] ‘ane un Jed gov[duial so uOT[WURYDE,| 
op ginal joued ynoy, ‘sdura} ourguI Ud saaUTWUIT9 yUOS US 
Inb ja ajgnbua | suep sogisgjur juos Inb saytun ,.p a[quiasua 
un 4so Joued up ‘sfoued sinorsnyd e1pussdwios uo][Hueysg | 
‘sdua} 9] suep jutod 3no} e ‘anb yuapusjUs-snos sgIgpIsuOd 
JUSUIOT[aANoue! op suryd say snoy, ***’4 39 A op sseuuoT[N 
-UBYD9,P SINDLID Sd] 9UD UOTWLIPLIOD LI] Op s[Qpow! J] 
Jasiogid suoasp SNoN ‘a1URIsUOD 3s9 ’a ‘aseUUOT[NULYDS Pp 
SINDUO,P O9s ek] op souvleaA ev] onb suosod snon 
“oureo 
-LIQUI® S gD PI Op svd 9] SULP BSeUUOT[HULYDE Pp SINOLIS sop 
SUOTJL[PLIOD S9] JOSI[QPOUL Op 39 IJOUINSS,p ONA Ud sapoyou 
SOSIOAIP JUSIPNID (0661) 997 39 (€661) SUCMH 39 Ja}sduiaq 
‘anbiso[ouosyo d19s op se[gpoul op usAoW ne ajgnbus.p 
SIND Sd] JOsI[gPOW Sp Issne JUdIQPIsUOd (€GG[) POIOG 
Jo FyezeI onb tsure ‘(966 1) JOWT[IH 19 [og “stuUQ-sIeIg 
XN& [ILI9P Ip SdIOUIUIOD 9] Ins seguUOp ap otI9s | INod uN 
juasodoid ua (€66 1) XOOIIM 39 [[oq onb sipury ‘aanoe UONR] 
-ndod ey ins suualyesysne ajansusw ajgnbuq ,| ap ssguuop 
so] Inod s[apou un juasodoid (gg61) [aed 3° [2921S 
‘giduroxe Jeg ‘eseuuoynueyse.p uejd np sayxerduio0s soy 
19}9[JO1 JOP Jo BLIgs e[ ap puadep uUONL[aLOD0INe ,p UODUOJ 
PR] Op astogid ULIOJ eT ‘21qQuIS Iso BSvUUO][HULYE,p SINdI 
-19 Pp dLI9s Bl OP UOTR[QLIOOOINV _p sINJONS eT Is stJI[duuIs 
9S JUSUWI[[2ANOUAI op suRyd SIOAIP Op JaJJO,| ap asATeur TJ 
‘Jaston 
yweunod uo | anb yuaWaT[aanoual op suv{d SIOAIP SAP 124J2,] 
* ayggor nb ajepow un ‘(44 |2 4) 4 Inod ‘11ulyep suoasp snoN 
‘a3dwi09 I1u9} usd Inod uonewNss,p epoyjoul oun gst[NN 
39 SjUeJSUOD JUSTe}I9 S[I.nb gsoddns jos ‘sduia} a] suep yulod 
no} & sjuepusdgpur jUsTe}9 sUOT[HURYdE saz anb gsoddns 
39 JUSWOT[aANOUaI ap ULTd Np a}dWI0D JIU} ap STWIO 110s U0 
SOOSI[PUUOSILSEP SOLI9S SOP VULLIVA P| OP UONPUI}sSd,] INs 
SINSLIQUe XNLARI] Op SINoNe Sa] ‘JUSWIATJaANOUAI op sue|d 
siaatp inod (44|44),4 sawiNse suoasp snou ‘oJat{nonsed 
ON] OVS ouN.p aseUUO][HULYE Pp SdULLILA PI INS JUS 
-9[[2Anoual op suvid sIOAIp ap J9JJO,| JOUTULIA}Ep INog 
‘afqissod aonb a0eorsjo Issne oseuUO][N 
-ueyo9 .p ueyd un Jasin ,p onbisoy wersed [I stew ‘ayypiqeuea 
op JueyOdu Indj}oejJ UN 3Sa o]]Ja Is onb oes eI Op dITIG 
-RLIGA P| B SANVOIJIUSIS UOSeJ Op sed onqiyUOd ou oseuUOTIN 
-URY99 Pp INIA | Is soyUeLOdUII Issne sed JUOs ou oseUUOTIN 
-ULY9 Pp ULTd 9] JURUIBDUOD SUOISIDEP SIT “JOOIIP JojJo uN B 
eseuuo][nueyoe .p ueyd ne satyelal UOISIO9p ev] ajanbey] ins 
a}uesodwiod P| INs SUOIUBDUOD snoU snoU “jUoUETJoANOUAI 
op uejd np xtoyo 2] swu0d ‘aseuUO][HUeYDg,p uetd np uoN 
-e10qe9 Pp SWg,qQoId sap Ins suoyoued snou snou onbsind 
STRUL ‘SoseULAR SUIeLIIO JojUasaid yreLINOd 91URI[NSEI OLI9S 


Z8l 


RI Joygidiajut Inod a7e10} souRUvA Pk] op aJdwi0d MUAY, ‘aseu 
-UO]]HULYIE Pp INdLI9,| op souLLIeA ey B UOMTSOddo sed a7e}0) 
SOURLIvA RB] Op UONsonb ev] suTUUeXd UO ([86]) INOsUO|] 
79 I9IJOM “2A & 91D0Sse soULIA PI ap aduI09 sed juan 
dU UGC ‘eseUUOT[HUeYSE .p uLtd Np UONPdTJIpow ve] JonTJUI 
ined aonb aseuuo][Hueyse .p soueLIvA e] Ins 1sa,9 onbsind 
‘g1uesOduI0d 33399 INS SUOIUDDUOD sNouU sNON ‘aseuUOTIN 
-ULYI9 ,P INILIO SUBS SLI9S 93190 WeAIOSQO UO |] Is WeIpUSNgo 
uo J onb Ang[eA eI 30,7, sdu9} ne oUTULID) as Inb suOTBUINSO ,.P 
dLU9s RP] Sgide.p onUd}gO SONI} INDTeA LI BIJUS DOUSIOTJIP 
Rl op 113e,s TT “A ‘a[[ae1 OLigs e[ ap sunayea xnev sgUUOpIO 
-qns ysa nb ‘44 /m 30 4d) m anua souslajyIp eT B a[e39 1S9 
do INgpea vk] op sseuUOT[ HUY .p Ina J “7 sdui9} ne 
dO[IJ INIeA L] op oseUUOT[HULYSE .p 9oUeLIRA P| BJUSSIdoI 


(Z) m(74 | O)yim= (14 |'O/ 
SIO[V 
(1) alm ='4 


Jed aguuop 3s0 7 sdurs} 
Ne sdI}[IJ INVA LT “19-d][99 op Ulj P| B 39 INGEp Ne sgtjIpoul 
ong juoAned sieul ‘alias el ap sdioo a] suep sjue}suO0d 
juos splod sa] ‘soeNTIJ sinapea soy] yuoyoddes as ojonbey] 
B D0USI9J91 op spotad ev] op puodap ’m oa8eNII ap sprod 
ap inojoea aT *’4 ayanbus,p suoneumnse soy Jaynoyeo inod 
sgsi[n asepuos ap spriod sa] daAe UdT] UNONe yUd}UaSaId 
dU 19 Salloyegye sed yuos ou oSeNLJ ap spiod sa] -’m a8enty 
ap sptod ap inojo0a un,p uoneoydde sed 44 op sinsed 
@ SINICA Sop JtU9}qO Inod gsI[YN oITeUT] O1N[IJ UN SUOIEpIS 
-uog “(14 |74)4 sed aqquasgidel ysa sfeulsu0 aL9s eb] ap 
aseUuUOT]HUeYE ,p soUeLIvA eT] ‘uONe[Ndod ap sayjaed sinaTeA 
$9] JUSTJUOD INb inojO0A 9] ‘4 q sed 19 7 sdwiay ne,nbsnf 
ajygnbua.p suoneunss sop onbisojouolys oLigs e[ op 
SINI]eA Sa] JUdTJUOD Inb Ina}dea 9] 4a Jed suojuesgidey 


AONVGNGL VT Ad SNOILVWILSA SAd 
Ld SAASITVNNOSIVSAd SNOILVIWILSA 
Sdd FOVNNOTILLNVHOG.d AONVINVA *€ 


“QOUdNTJUI 9UN VoJOX9 SseUUOT[HULYOY,p UL] 9] o[Jonbey ins 
SOQUITISS SOLIDS SOP SI]IGvURA ] 9p aJURSOdUWIOD eT SUIZDUO0S 
Inb 99 Ua S}e][NSaI SINI|[IoW Sy JUDUUOP INb JUuSWIT[aANOUAI 
op suvjd sap sept oun elouUOp apnig 91190 “sajURSsoI9}UI 
sgeudoid soureyie9 Jajuesgid juateunod ‘ayjonjoe onsy| & 
sasijn sed yuos ou sjI,s ourgu ‘Inb yUSWST[oANOUAI op sueyd 
Op SIQUIOU UIe}II0 UN B 39 WqINA S92] Inod a]janyoe oinay | 
B Sgsi[iyn JUWIsT[oAnouel op suejd xne agID90sse soUep 
-Ud} B] OP SUOTRUITSS Sap 30 SoasI[eUUOSIesep SUOTEWINSS 
SOP IBLUUOT[NULYIE ,P DOURLIVA PI II SUOIOPISUOD SNON 
‘(S) [-Z-[ JUOWT[eANOUAI op uryd 
DOAB o][aNsuaU ajgnbus oun ve said ned e puodsa0s sanoe 
uoneindod ey ins enbruuejig a]jaLsour aygnbue,, aonb 
9AIBSQO & (1661) 1921S “(Z661 SAV) BHeNsny us snbrjdde 
mnjad & ‘B= MW NO IN[Id 19 (OG6I ePARIA 10 OUIquIeH 
‘MOIq ‘YB8UIS) epeueD ne 9SI[4N JUSWOT[eANOUSI op ue{d 


0002 eiqwesap ‘ajenbua,p senbiuyoe | 


ne puodsai0d g = Ww NO sed FJ ‘UOT[HULYDE,| op soUTUITS 
yuos sj! tonb sgide stoul w juepued snjour juos sguuon 
-d9[9S SJUSWIASO] So] [onbe] UoTes w-inod-supp yUSUIdT[OA 
-noual ap uejd un yuango uo ‘Q = q asod uo, IS “(7661 
youvsuex 19 Wepy ‘Jo[[N4) (g)p-g-p ued un uoles 99) 
-Nd9X9 189 (SqQ) Kaaing uoynjndog juasinZ ej ‘stup-sierq 
XN ‘JUoUIa]JaANoUs! op suejd op sassefo das inod 
soluly uoNejndod op xnvj0} sap yo souUdAOW sap uONeUl 
-1]S9 | JUSIQPISUOD (POG) WeYLID 39 ORY ‘sUOISeID0 wu INod 
‘1NO} Ud ‘SaSNIDUI JUSIOS SagUUOTdETES soyIUN sa] aonb 910s 
op siedqi so ueld aq] ‘stow v ap aireyuawigyddns aporied oun 
Inod uo][HuUeYsg | suep soIINpoHuTaI sind ‘stow g yuepued 
UOT[NULYD9,| OP SdgIIOI ‘sJNDgSUOD sO! H yURpUad sasnyo 
-UI JUOS SagUuUONdEIES SgITUN sa] [onbey UOTas (W)0-g-v JUST 
-9][2Anoudl op uryd np xnerloeds seo sap uos sgidope juoul 
-9[[2Anouel op suryd soy ‘onbyeid uq “(WqINq) eAvoe uo 
-e[ndod ev] Ins sajjansusw! saygnbus saj Ins tor suomUsoUO0S 
SNOU SNON “SUOISNIDUI So] JUS S|[PAIOJUL | B Jo aJanbua,| op 
uOT[NHULYdY | SUP dSN[dUT Iso 9}1UN oUN Nb sIOj op o1quIOoU 
ne jel] & nb 9d Ud JUDLIVA JUDUIAT[9ANOUAI op sueyd sa] 
(€661 ONID 10 UOey 
JOA ‘Soulg[{qoid sao ap UOISsNdsIp ouN INOd) sal|[IenseI 
aguuop sop syenb ev] 39 asuodal ap xnv} 9] Jassieq ole} op 
anbsi uoljeAlosqo .p sopotied op o1quiou pues un yuepued 
uo][HURYOS | sup o}IUN oUN IIUD}O1 Op yey a] ‘anbiyj9.p 
suosiel sop Inod asuodgi op neaprey 9] Jedi ap ayIssoo9u 
RF] aNO “UoT[HULYSE | op soUTWIITS ong JUOWTeUTy WOp 
BQUUOTIIETIS 9}1UN 9}N0} “URpUddaD ‘ssANdgsUOD sapotiad 
ap g{qissod anb snZuoj Issne ates oun yuepued osnjoul 
jso sQUUONSeI9s 9]1UN oUN sjanbsap NYVA Ud JUSWII|[9ANOUSI 
op surjd sap ouop i] qQeI9 UGE ‘aJgnbus,| ap INOd JI WNpel 
apoudsd onZuo] oun juepuad uoT[HULYDg | SUep SagUUOTIDE[IS 
SONUN SI] IUd}OI Op We} VY] “asnaynoo snjd vy sa mnb a][99 ys9 
ajgnbue | suep sauUOT}de[9S 9}1UN dUN,p UOISNjoOUT sed 
PR] JUOWT[anIqey “a][enuue UOeLIeA ek] Sp SUOTVUINSE Sop 
SBCUUO][NULYIE Pp DOULLILA P] JINPE SIOW Z| Op I[[BAIOIUI 
un Jed sagiedgs sapouied sop onua jueyodui juowsys 
-neaoyo un.nb sipur} ‘ane | & spoltied oun,p UOTeLIRA P| 
OP SUOTLUNSS sop aBeUUO][NULYIE ,_Pp SdULLIRA PT JINP9J SOAN] 
-Ndgsuod sapolied xnap ap suOT[HULYIE so] aUS JUBLOdUIT 
JUSWUDYONeASYS UL) ‘sInd}oRJ sinaisnjd ep puedsp juow 
-d]]2Anoual ap uejd np x1oyo ay] ‘aouepus} ke] ap soyeursLO 
SUOT]LUINSA Sap Jo SagsITeUUOSTeSEp SUOTIRUIISS Sop odeU 
-UO][QULYD9 Pp SOULLIVA PT DUOP ‘aseUUOT[NULYIY ,.p SINdLA.p 
OLIQS P| Op UONL[ELODO;NV ,p aiNjony)s ek] VoUENTJUI sJgnbus | 
Inod tstoyd uoT[HULYD9 | ap JUSWIOT[oANOUAI op ued JT 
‘gSI[NN Jotfnonsed yUsWaT[aANoual op uryd 9] Inod 
SOQSI[PUUOSILSSP SUONLUINSO Sop SOULLIRA B] OP SUOTVUINSS 
Sop oINpold & JUDWIdT[ITJUISSO JUSTIA Jo JUSWA[[aANOUaI op 
sur|d sIaAIp sap 1aJJ9a,| ap o}duroo sed yuoUUIT) OU XNBART} 
S8D ‘SaTQpoul sap sajguiesed sop UOTeUINSA,| INS JUSI}UIS 
-U09 9S 19 ‘aBLUUO][NULYDY,P SINILIO,.p aUI9s Re] B Nb a[[99I 
OLI9S PI B UII Issne saoI[dxe WT Se[gpoul ep uoneoryd 
-de | quorpmag “(¢g6 1) Uosie AA 30 URUUSNeH] 19 (C861) SHTEAA 
19 o3plung (7661) JONLL “(€661) XOOTEM 3° [109A SUTUIOD 
‘sinoine sonne.q ‘anbiZojouo1ys otigs e] JUoWOOIdxe 


sed jUasI[gpoul aU SepoUIeU sa_g ‘uoTLOII[duIIs ap sasau] 
-OdAY SasIdAIp JUR[NULIOJ Ud ‘OgUIT}sa aISOTOUOIYO sLIVs eB] 
sqide .p JUSWID}DeIIP OseUUOT[HULYDE ,p INaLIa,| sp UoNeWIN 
-so oun allnpoid ap asodoid (p66 ]) WURULIAIJOIg “[ [xX apoy) 
-9UI P| ap olegul] UoTeUIxosdde oun Ins sexe a[qQe{quies 
epoyjsur aun aydope (C661) JOINS “soURLIBAODOING pp 
UOT}OUOJ L] INs SaMVQUI] SAN[IJ Op 39JJO | INS sappuoj apoulguT 
aun juajidope (1861) INosuoJ] 32 J0}[04 onb stpuey ‘aseu 
-UO][HULYS9 ,.P SINILIA Pp ILI9s Le] Op oNd0ds 9] INS souTegUTT 
SONI SOP JOJJ9,] JUSIQPISUOS (¥QGT) [PINEC 30 eas ‘old 
-wexe Jed ‘sagsodoid 919 1U0 sopoyseu sasJOAIp ‘pre39 199 
Y ‘Slal[NZaUI sap jo gU[euUOstes ve] ap ‘souepUd) eI Op soa 
-RUII]SO SILI9S Sop ISVUUOT[NULYDE Pp DOULLIVA P| JOUITISS,p 
yOULIod SUOTIBUNT}SE S9d ap [NI[vO FY] ‘esvuUOT[NULYDE,p so} 
-IUN Sop neaalu ne sJgnbue.p saguuop sey side .p ‘’a ‘adeu 
-UO[[NULYI9 .P SINILIO _P SLIVS BP] Op PoULIIBAOD Op aIMjONNs 
BR] JOWIN}So jned UG ‘soIgITeSOUI Ja OURpUD) op ‘sargruUOSTeS 
SojURSOdUI0S sap UOTJEWINSO | INS onTJuUr ‘Ino} UOs ze ‘Inb ain} 
-onns ‘’a yo '4 SaLigs sop uoNeTeuO0o0jne | red sguTULIa}9p 
ISO S9AIBSQO OLIVS LI OP UONL[aOOINe _p sINjONYs vey] 
‘uonsonb Ud SLI9s PI OP 9I[NR[OA BP] UOTAS SOULIA} ¢7 Op NO 
€1 Op [IJ UN JURSSISIOYS Ud ‘HIJA Sop UoNestNN | JUep 
-ULWIWU0DAI SII ‘SaT[ONSUSUI SdLIgs So] INOg ‘SeT[etotJo sanb 
-IIS9R}S SLI9s Sop alinpoid ap ona Us soULpUd} Le] op suOTeUT 
-1]S9,.p SSPOYISUI Sap IPN 1U0 (166) AUUSyY Jo SaTMOUy 
“9UIVYNOS PII]IQIsUaS Op Jo SBRSST] OP S1IZap a] UOTAS S]UaIETJIp 
SaN[IJ Sop IIstoys ynad uo | onb 39 onbrun sed sau souepus} 
Rl ap uontulop ey aonb sanbsewios U0J (7861) Ulqing 39 
Auusy ‘sagttqnd sogstjeuuosiesep SUOTeUITSS xne SON[IJ SOT 
quenbijdde ua souerpus) e[ ap suoneuse sop ortnpodd Issne 
ined saLigs sap InayesiqyN,T “HINA Sep [Noyes 9] Juenbrydxa 
(9661) uoswoY], 19 AvID 39 (786) Ulging ye Auusy ‘uoN 
-eSI[BUUOSTeSAap P] Op SUI} XNe SdLIgs Sop sdURpPUd} P| SITe.NXS 
inod WAIYV7CIX 1 VWINVIIX ‘TTX suep sogsiun 
UOS ‘xneLIenjoe spnojeo sap nod yedep ne jutod ne sos 
‘(9161) UOsIapueH{ ep Se[tqoul sauueKoW! ST (E661 SAV 
‘1661 Quez 2 aloeUIT) soULpUd) eB] Op SUONRUINSS S95 
Ins JUeSeQ 9S Ud SLIVS B] Jo}QIdIOIUI B SINIesI[NN so] oseIN0S 
-UD 19 V9SI[PUUOSIESEp OLias BP] B HINW Sop uoneordde sed 
SONUs}qgo VoUEpUd} FE] Op SUOTIBUINSa Sap Issne aqnd S_yV 
“VININVZILX aueuea ev] & sagid9}UI JUOs Inb suonesorourE 
sane Pp IWOsp 3UO (S661) Y8Ng 39 [9G “ONO ‘TesuoW 
‘A2[PUL “(S86T 3° 0861) WNSeq sed sg10qeIp VWTAVIIX 
QULLIVA ef 19 (L961) [109 39 ULysIYS Jed yurod ne ost 
LLX snsusd spoyisul ev] JUOs yUSWUeINOD snd 9] Se9gsT] 
-IIN SapoyIeUl Sa] “saIegUIT SOI Op UOSTeUTqUIOD oUN ._p 
udXoul ne yUdUIaTTaNIIQeY ‘aLI9s BI op JoUTUNITS,[ ep 12 ‘g 
JOUINS,p JUBAESSA UD SaQSTTRUUOSTeSEp SOLIZS Sap JUasInpoId 
anbnsneis e] ap xneoing op siquioN ‘sgudoidde snjd ang 
jnod saneordnnur uontsodwiosgep oun ‘seo sureyeo sued 


Shr ety deck ir 1 ='4 
anb 
uarq is “7 19 'g “7 sdorn3gu sinof sop 39 gujeuUOsTes e| 
ap ‘a[aXo-aouepus} e| op sajuesoduw09 sa] puesduro09 '{ ules 
e] onb alepisuos uC ‘a8euUOT[NULYDE,P INeLIO,| 189 ’a NO 


eBeuuolnUeYoS,p BOUBA B| INS JUaWA||BANOUAI Ep SUL|d SJOAIP OP JOYS,7 -|28IS Je UBIE TOW) 981 


“ne Nps MON ® [ee}sp ‘ne npo Mon @ dW TeIO 


sanbuonse|g JaWINOD “elfeNsny ‘77ST MASN ‘“SuOsUOTIOAA JO AiIsIoATUA ‘sonsHeIg polddy pue soneusyjey JO [OOYDS ‘JeaIg “O'C 19 UaIeTIW “HD , 


‘) + iS = ™ 
sed ‘'x ‘aruty uonerndod ev] suep auigs eb] op a][99e1 INdTeA 
RI & Salar isa 7 sdura}_ nv agArasqgo INgyeA wT ‘sojodo1 o8ep 
-uos sed ajgnbus oun,p quouuataoid y ‘[ = 7 ‘7A samopea 
SQ] JUOp s9LIVAIUN oNbIZO[OUOIYD OLI9s SUN SUOI9PISUOD, 


INANATTAANONAY AG SNVTd “7 


“QOULPUd} LE] IP SUONLUINSa Sap 9 SoasITeUUOSIesap sUOT} 
-PCUINNSS Sap ane | & Opoled ouN Pp UOTVLILA PJ ANS 39 NEAT 
QJ ANS IOI BIUBOUOD 9S UGC *(YR6I 19 OXI) WINseq Jed juIod 
ne ost VWTAVITX epoyjour ey red 19 (1961) oavissnyy 
39 Buno X ‘urysiys sed yutod ne as ([ [x snsusg spoyjow) 
neaing snsusd SN Np spoyjewl ke] op [TK aueLeA ey] Jed 
SONUd}QO BdULPUD} EP] OP SUOTJPUIT}SO Sap 39 SogsITRUUOSTeSEp 
SUOTLUIT|SS Sop aseuUOT[NUeYDE,p DOURLIRA eT INS ISTOyD 
JUSWI9T[SANOua ap UR] Np s}ajJo So] JOUTWLID}9p op so opnig 
ajuasgid ey op Jnoalqo J ‘suo[[HUeYye sap yUoWIOYONeADY 
9] eOUTULIO}9p Inb ‘sdura} np [I ne ajgnbusa.| suep ssgr[no 
-tued 911un oun .p UONeIsE\UI,p ueTd 9] “Ip JUoWATNe “UU 
-9]]2Anouel op urd 9] 3s sgjadei aygnbus oun .p sseuuo][N 
-ueyod.p uvjd np uonesoqeys.| ap Januasse yoodse uq 

‘(8661 USE) SPANNIQSUOD sopolied xnop 
aIjUS UONRURA PI Op SUOTFBUITISS Sap 9dA}-11899 | 19 SINOD SO] 
dIINPel Op ONA US oINNe | & spoted ouN.p oINsoU! sUIeLIOO 
oUN SURp JUSYONRADYO 9s sUOT[NULYDE Soy ‘TeIQUES Ud ‘sTeU 
‘sdur9} np [IJ ne sjuepuadgput suoy[nuLyse sap Ins sogpuoj 
JUOS SOLI9S So] ‘SBD SUTRLIOD SUR] “OSRUUOTINURYSE _p INdLI || 
B INP [9d JUSANOS 3S9 SdgSI[BUUOSIeSAp SUOTFBUIT}SS sop 
OULLIRA P| Op aJUeUTWOp a}UesOduUIOD FI ‘(666 1) Jowery 10 
[[9q WeAINS ‘oseUuUOT[NULYIE ,_p sINaLIa soy Jed sagouonjul 


JUSUTITVZ9 JUOS DOULPUD} Pj] SP SUOTUIT}SS Soy 39 SogsTTeU 
-UOSIS9P SUOTILUINSS Sd] ‘S[VUISIIO SLES PL] B SJUSUIO}IEN 
sulvyiood op uoneorydde sed yinpoid say uo nbsing (L661 
SO[MOU) SOPOYISU! SISIOAIp Jed souLpUd} P| Op SUOCTFEUISA 
sap juasinpoid onbnsnejs e] op xneyUOWUIOANOS xneoing 
sane. .q (L861 SAV) Sansa sayuesoduiod soy JOsst] 
ud Inod sagsi[euUOsIessp soligs xne (HIJA) woslopucH 
ep seiqoul souusXow sop spoyjoul eT op uoneordde 
ed sanuajqo s0uepud) &] Op SUOTPUINSA Pp SoLIgs sop gI[qnd 
® (SAV) sonsneis jo nesing uelensny | ‘sue xIp op snjd 
JULPUD “SIAL SUIQUI NP 1S9 (1661) YS ‘sogstyeuuOstesop 
UOU So][ONSUSU SINIVA sep salgyNseM suonenjony 
xne nb sajusoel-snos saourpus} xne a8ejJURAEP JUDSSOIIUI ,S 
s[i.nb yUuaanos yuoiejoep sonbryyod ep sayskyeue soy 
‘(Z861) ulqing 319 Auusy UoTaS ‘sogstITeuUOSIesep soguUOp 
op soaliegs sap sduiajsuo, sindap juasinpoid onbnsneis 
Rl op xneyUOWsUIZANOS xneoing sop yednid ey ‘sags 
-I[euuOsIesap Solas sop osATeue Jed ssouepud) Sop no uN 
-BLIVA Op sajusdel[-snos saqiInod sap UOHEN|eAY | Ins sn{d us 
snjd op aUd0U09 as sanbiZojoUoIYD sollgs sep epnig J 
‘aygnbus .p suoneuinss .p onbisojouoIYo s1I9s ev] op 
DINI[IQVUeA FP] INs oNIJUI ‘sUOT|HURYDE Sop JUSWIOYONeADYD op 
ueyd 9] Jarpnonsed ua ‘aseuuoy]HueYyog ,p ueyd 9] ‘ouog ‘aitfIq 
-RLIVA Op B]ULLIOdUIT BdINOS SUN JUOS SINdLIO S99 ‘soLIgs Op 
diquiou Inod ‘19 ‘aseuUOT[HUeYSE .p sINdLI xne ‘ated us 
‘ual saguIss sonbiZo[OUOIYD Salas Sap dy]IQeURA ey] ‘aseu 
-uoT[NUeYDE | op NUD) a}duIOD ‘axaTdwI09 Js9 aNe | B OpOL 
-od oun .p suo] UeYdE sop JUSWIdYyONeADYD op ue] 9] yUOp 
sogjodoi o8epuos sed sajgnbusp saguuop say side ,p sajinp 
-o1d juos sayueyodurt sanbiso[OuoIY9 satis ap dIQUION 


NOILONGOULINI ‘T 


‘JUSUID|[9ANOUSI op uLTd ‘soURpUd} BP] Op UONeUISO ‘UOTeSITeUUOSIeSep ‘VWINIMVILX ‘IIX ‘S10 SLOW 


‘asepuos Jed sajgnbuoa sep ins sagpuoj sonbisojouo1yo 


SOL9s sop sgide ,p sag[NoTeo BoUepUD) PB] OP SUONLUINSO Sop Jd SOVST[PUUOSTIeSIP SUOTJEUIT}SS Sop SdeUUOT[NULYOP ,.P DOULLILA 
RP] INS suUOT[MULYDS sop JUSWIAYONeADYS op sue[d SIDAIP ap Sjojjo So] WOEp poe JUasgid a7] ‘souRpUd} |] Op suOeUIN)SO 
sop oinpold op Issne jUa}}0ULIAd ‘sogstfeuUOSIesep SUONLUINSO sap olINpold Inod JusWTUTeINOD osI[yN UO, anb “‘WINTAVITX 
sWIUIeIZOI1d 9] 19 NeaINg snsudsD Sf NP spoyjsul e] Op [[K oUeUA eT ‘sanbisofouosyo satigs seo seide.p sayimpold 
OULPUD} LE] OP SUONLUIT}SO So] Jo SogsTTeUUOSTesap SUOTJeUITSA Sa] onb Isure ‘ajgnbua,p saguUOp sap INs sagseq sUONLUINSS .p 
ONbISO[OUOIYN OLI9s L] Ip SI[IQeUeA ke] DOUSNTJUI oIINe | e spouad oun,.p uOT[HUeYIE | op JUoUIAYONeADYO oJ ‘WUaUIgsTOgId 
snid ‘onbisojouosyo ous e| op sg39Udold sa] souantul oseuuo][HueYoe ,p uezd 9] ‘IQ ‘oxa[duios sseuUOT[HULYDg,p uEyd e 
asepuos Jed sajgnbus sap Ins segpuoj juOs sonbisojouoIyo sotgs sasnaiquiou ap ‘anbruouos9 Ja [eI90s souUTeWOp so] sue 


aWwnsaa 


(THA LS (9° 9? NAUVW' PW “HOD 


IIULPUI) L] OP SUOIVUIT)SS Sap 
}9 SIISI[VUUOSILSIP SUOILUISS Sap dsvUuUOT[WULYII,p 
IIULLIVA BL ANS JUIWI|[IANOUIA Vp SULA SIIAIP OP J9IJ9,'T 


S8l 


epeuey enbiysieys 
G61-S8t “dd ‘Z ,u ‘9z JOA 
000z e:quwscsp ‘a}enbue,p senbiuyos | 


arena a 
ig aoe a ¥) Cherry 


wie ay a . i 
63 “Widens | me 
iipbak ¥ ies ie a, ew Eisay Ga aki 
: ce y ed aed Th fi io eh | 
Caneme whe vin ak Ue on Bt oTA 3 hi 
Recent eae Ade we Asan aan yy os fh 
Bal hhh AGN Le ie a ined uae iy pith Abin 

ably aly ete CTE aN v Ee apn 

At ieneteet eeeeclibedan, vlg® RO. pW ee Oe la ta 
CaO ui ane a Pan to eye yee? 

Sapam Pa ALi 5 appl hauler” onthe 
eit Pe ! aetr Reason Fate vlan breve (SAN eae SPSCmPs, OS Un bn 
tn » Shvniohey Lig RPE AOE OR eae? TU SRE Rac eee. 
mye Pei, ately Rr, r7 g LG tne + eae as a) ( oy) are ; 
saree ager Trt Mb; i “t Satie ets ns i ny - Pi. wet 


= Prius) inh ay 


pea ede naeen: qed oad Mey angels oe Gur. arch didn SIA bes 


RNG FAOSENED (ARMY Tar Gi tae Gee it Ma Uae Des ere ; 
Sea eh PHA SESE Ge daRa Sty Ey ROA IES Sel Se nan 
ents i ea (0) A, Ue hha At AT We 


VPC ery Pais) A) MS 5 f wot ee ‘ 
a 7 - 
e : PF - . abe hae? "te SAGA he wy b> 


> ; nd ’ Ay CR h apes 
j a 9 M t t x ia, 
™ . A iat ‘ KV A} * aA f a: i ant Diy Mee FOR 
ir 7 ™ fT af 5 a ~~ 

; 

‘ ee de uf , 78a wyiits! 
" .! * 
7 i Ke m cove & £ 
rape ’ , oy 

‘ a ’ Jey a ne 


- *; ie ene seieaaitl. yoo * a l | ele pre ; aS y 
oe. i, eg gece De a ea it Ne an i Nala alee ai 
a , a? vil > kel f , i ; ; * ” 


PS ie Kees , 0 ; / ; “t eqers 7, 1 iv 


call 
4 
cv) 
- 
i? 
+ 
— 
= 


vip often: r ~— = 
5 poaebeinrna ect f i. 
ig TM. hig ba tmierin a. 4 . dinecaegpeene Sy eae are 
Bo ete meiatajrr fies a rigs T ne KK. ai Ot ore eA rr Ar: Bea” Ye 
agement : bu Th Le bnyth Gd. 4 uit 

sgh fn of } ray “dpc at ess 7 tray ' it in a4? cut’ ee bid won 
Le regions 250 Ute. Gord al OSs Sto : . ‘ a 

BEAN PRNES TORE, te ee ee ; 
Be cease c/ayre oe 1) were he om + sae, Hn Sapam ona 


4" ’ pexbuirnasy aif Larne ps iy  rieuer . rl ape ery 0 Pe > ee Ane 1 ee Al eres ir ee hia is 5 eee thins 
at Ng Sie mpi Tu. me A ke | ee cA 4 j . : 4%, 2 pe ut ne OF eeskie = we : 4 Mh ad! 
ee oe ae at 


eysis vile Oe ee ee iY" coed te GB Gi 1 nttep neon dee eS 
a reg & enbbiee ue ciaecs Sete Owibea chon ce ae oe Geely arr Mer BCS ares 
reat alecstodi aii sti white 1 __ PeAanuahar sa a bee Se I IC 
‘Kee ‘ : rik ous et) tal e-vicau atta? TREES Gc em wee id Oo ag 
riety > aU ‘ s toil hone > ri ud eevee Tots bt ae 1 r ’ satin 
* i - bet n ' stile ~— * “i ituicirren (Ete Gli Piel wae ° 4 eee] Wh He, yeye rr; 
reerre Le Hity, wd: aa eel | J : i ‘ oa ore Z ’ ay ast 4 ’ ; Sat * 
, : J . , is ew ven an LA. oa "oe 
mpaitis ectul ay Pal 0 ose acy te” Uae Osea Lee é pomleeaae $0 SRE Bi OP 


/ a euieea! sry age qi er SC en mm) o- Pa Cs a 7) oF leS tps Ty COME 
mI ak nut fog Bay eR quelli 2 Ae 


& ere Ty ai , le - : ‘ 
A } , ’, SH wer y : ) | 


oo 
ee et ‘ od 
Ue a ‘ ® 


—" a ' SN UREA Ngee i > ; ti oe pangs, wu | re ? fea] 


‘9P9-SE9 ‘Lp “UOIDID0SSy 
poousunig unouaup ay fo joudnor ‘sainseaut uonIsod s3y0 


pue sUeIPOU! IO} speAIoJUL sdUEpYUOD (Z7S61) ‘SU “SAHNUGOOM 
“UY ‘8eIIO A -JosuLIdS :yJOX 
MAN “UONDUIIST aouD1dDA 0} UoNINpo.Uy “(C861) WA YALTIOM 
‘(UOISIAIG SUTUUILIZOIg pue spoye 
Teonsnerg ormouosg a[qruodstp *[0g6-WSa# enbruysa) woddey) 
“snsuad au} Jo neaIng ‘$"q :Od ‘UOISUTIYsEAA “(DOS) uonon4suoD 
fo daring ayz sof saouviany suyowuysq of uoyvoiyjdey 
ajdug-fop] payfipoyy fo uouonpoay (8661) ‘fA ‘NOSdNOHL 
“BPE-EVE (16 ‘UONDIDOSSP [DI1IS1IDIS 
updisaup ay fo joudnor ‘Surjdwies wopurel poynels Ul UONeUINSS 
sourLeA o[dures-sfey peouefeq UO (9661) ‘f ‘OWHS 39 “M'N'f ‘OVa 
“PET-STZ ‘BI ‘ajanbua, p sanbiuysa J ‘soxo{duiod saygnbua 
xne sofqeordde a8euuo][nueyogel ap sopoyyoul soy Ins syus99I 
xneaen sonbjand (7661) “M “ANA 2° “fA ‘AM “WNL ‘OVE 
‘uy ‘suog pue 
LENIN uyof yJOX MIN ‘sanbiuyza J uoljDjNuUy dajnduo) (8961) 
“I ‘NHO 2 “Ss ‘d “Morand “Tf ‘AALNITVd “HL UYOTAVN 


‘QUIOIUL UOTEIUSUINDOp snsua_ oy} JO neeng 91[qnd uou oysneqy 
‘dadpg ooriuysay uoijonsysuoy fo daring (0661) ‘W'd ‘AYANT 

'6S-Lt ‘€Z ‘atanbua, p sanbiuysay, -onbuidurs 

epnyg - nusAal np uoNesuelod ek] ap Jo 9ilfesguI,| ep somnsoul sop 
souBLRA B| op UOHeWINSY (L661) “AA “ONNA 9 “W ‘OIAZOVAOM 

“Sp-ST ‘QI ‘anbisijojs ap auuaipupuvo 

andad DT ‘SayeUullysa AVAINS UI SION IINSBOUI 0} SPOYyJOU! JOY}0 
pue densioog (8861) fH ‘NA 3° “WN ‘OVE “OT ‘UVAON 

‘6ET-E7T “9 ‘SousuDIS po1nGO fo jouanor 
‘UONeUINsa souRLeA Joy poyjoul s Ae (0661) “Ud ‘SNIMaNL 

“6VII-LPII “€8 ‘MonNDII0Ssp jooNSIIDIS 

updidaup ayl fO JoUANOL “BUITOQR| JOI[INO JOJ Soa[NI jULIsSISoI 
owos Surun-ouly (1861) “A ‘ZOIMATOI  “O'd ‘NITOVOH 

‘yoday snsuag 

_ 94) JO neaing giqnd uoN ‘uoNeUOWINI0g WreISOIg ‘skoAINS 
x[dwioD Joy uonewNsy sourueA =X TdA (S661) “AU ‘AVA 

‘UOLIDIIOSSP J0I1j81JDjS uDIIAaUP ‘SpoyjaP YoAvasay 

Aaaing uo uoijoag ayj fo sduipaaz0ag “suoeNo[ed soueLeA 
JO} SuNYSIOM ayeot[dai Jo uoNeodde pue Asooyy, (6861) “AY ‘AVA 


‘ouy “Buryst[qng Aa[sap-uosIppy 
“VW ‘Bulpesy ‘sousuoig pun A17190G04g (9861) ‘W ‘LOOYDAA 


GIHdVYOOITsla 


“UOISSNOISIP B] JO[NUINS ap 39 
SPSSIIQIUT SINBYDIOY SI] JOUOJUI Pp aNjouLed yop woddes 
jussgid Np UOISNYJIp ey] “snsua_d oy} Jo neaing np sal[aroyyjo 
suoneorqnd say onb xnatjnur suroul UsWeXe UN Igns B II 
“snsud_ 94} JO nesIng np jeuuosied a] Jed sasudayus sas] 
RUB ,P 39 SOYDIOYIEI Op sjeI[Nsa1 Sa] WWDep gsodxe juasgid 
YJ ‘o[BUISUO UONe[NUIIS op apnig,] Ins sapyN sonbseulal sas 
op ory “WN’f onb tsure “Wosnuew juasgid np soinati9jue 
SUOISIOA op jolns ne sonbseUlal sasna‘gUes SINZ] Op g1d0sse 
Ina\yoepgl sf 19 sourAUOUe sINaJeUTWIeXd xnap onb sulgUI 


est 


ap ‘stup)-sieyq Sap snsusdd oy) JO neaing np uesey sower 
yo Suenyy yoqezi[_ Jools! eB JUOUUT} sInojne so] 


SINGNGO WANT 


“UONTadaI SOAR DOURLICA PI OP 
UONCUINS Pp SdINPQI0IJd SOSIOAIP Op OpIe | & I9PULUUOII 
ainpgd0id ke] op ossajsnqol eI JURAIISGgO US 39 JoIse 
OP SO][Ie} sone ,p JULUTWUeXd Ud ‘oURIPSUL PI op SUOTFEU)SO 
Sop UOISIOgId PB] 39 UOT[HURYDE,| Op o[Ie} eV] oUS UOTRIOI 
RP] JuvUTeXd Ud a[dWoxa Jed ‘oyoJoYoeI RI aJAINsInod 
yeunod uC ‘seguUOp op saTqUIdsUd sone Pp ddAe dINj}OOf 
-U09 93199 ISIJLI9A SP INS USIG JUSIAUOD [I SIeW “UdWOTe3O 
SONUUOS SUOTINGLSIP sone .p Inod Josi~e1gugs JUsSSIe] 
as s}e][Nsg1 soo aonb suowisa snoN ‘(sanpudA suoOsTeul 
Sop 9]U9A op xud 9] Inod surow np) SH 2edA1 ep uonneder 
DOAB OURLIVA EL] OP SUONLUT}SS SAINT[IOW Op 39 SUOTJEUUIT}SO 
SaINd][ISW ap ouUOp aI[9 ‘UIJu_ “saygnbus sap jusW9yIEN) 
op gslje1gues [SIOISO] UN eB JOIS9JUI JUSWIOSIe JUSSI] 
as saguUOp sap juapuedep Inb sayjeasajul soy “JUOWOWII 
-SIOLL, “SUO][URYD9-IWAp op 1} J] B}IA9 UO] onbsind senby 
-PCULIOJUI SDINOSSAI $3] JASTUIOUOIY ,p JOULIAd dI[a “JUUIOUIT 
-xnoq ‘uoysenb us uonnqiusip ej ep puadsp souigwi-xno 
SIOISeO Sop Inesse] eT onbsind ‘suomnqysIp op xtoyd un 
Inod usaiq suuONoUO} aq “a1qQuIdepe so a[a ‘ploge ,p no], 
"SOBEIULAR SINIIsN{d djJO VppURWIWOSEI Spoy}SU eT 
‘uoT}ONSUOD Jo AdAING eT Inod suopuvul 
-w0de1 snou anb ai]Ja 389.9 19 ‘SHI edA} op uoNnNedgi 
gun Inod ‘aoueLeA P] Op 39 ULIPSU 9]U90A op xUd np suon 
-PUII}S9 SOINS|[[IOW So] QUUOP B ([k10} Nv SIOISVD [p) BOULIOJ 
-suel} a][OY99,| Inod ¢Z InasIe] ap SJoIsed op uOTeSI[NN TJ 
“QOULLIVA BP] OP UOTLUINSOINs v] JUOWOpULIS JINpel spoyyoU 
91199 ‘Jsa snjd Inb a_D ‘ainsaul OBR] ouN sup JUOWATTONIIQeY 
‘$}e][NSOI SINI[[IOU Sd] 1WINOJ 1UO (SeguUOP sop JUpusdap 
Inb JaIsed op sinasie[) SoguULIOJsuel] saguuOp soz uUOTAaS 
Sag[Odiajul souPIpeUl sa] ‘ssgtnuIs suoye[ndod anenb sey 
sueq ‘SHI 244} op uonneadal d0Ae S0URLIRA P| Op SUOTIRU 
-I]S9 so] Inod sueipeuw e] op UONeUISa ,p soinpgooid sa}UdI 
-9}JIp sap soiatidod soy Jourwexe p stunted & uONe[NUIIS op 
2pnye WON “SHI 2dA) op uonnedas ey inod soueea ey ap 
SUOTJCUIT}SS Sop Ins sgnbiew souaNJUl ouUN ddJOXe spool 
R[ op xtoyd a] onb yuonbrpur sanbuidura syeijnsei SON 
“S9QUUOP SOP UOTJOUOJ Ud JUdSURYS UOTNQL) 
-SIP BI SURP SIOISVD Sop JUSIUIOe [CWI | 39 INDBIR] VT ‘SOUT 
-1OJsuvJ) SOQUUOP Sop ddUaSgId UY “dI[NOTJJIP 91190 JOBLUOD 
g[quies ev aftWenb sWIgISION) J] Ins sapuoj oIIegUT] UOTeUI 
-lojsuey eT] ‘sduwi9) np [I ne anos uo][WUeYd9 | op uo 
-nqiujsip ev] onb oinsowi B JosueYd jnad ja auIvUIOp 2] UOTAS 
JUDUI[QVIOPISUOD JOJOJJIP ined ‘aireniqie ang ynod ino3 
-IR] 99D ‘plepurys InaBIe] Op sJoIsvd US SanuIUOD saguUOp 
op jUsUIAsSse[O UN asoddns sIesUT] UOTe[OdIA\UT TJ “oITeSUI] 
UOHR[OdIDUI.| 39 DJDIIP UOUIT]SA,| :soNpUdA sUOSTeUL 
SOP ULIPSUI 9jU9A Op Xd np UOTeUIT}sa,p SepoyyeUl xNop op 
SUOTJLLILA SOP B SINODAI NP JOJJO,[ QUIUIEXS SUOAR SNON 


NOISNTONOOD ‘9 


000zZ e:quiesegp ‘aygnbua,p senbiuyoe | 


“QOURLIVA BI OP UONLUINSA .p Jo UOeUINSa .p sayatIdoid 
sap iso Inb 99 Inod oueIpoul ke] op UOTBUINSA,p ainpgod01d 
SINI|[IOW LI WOS CZ INosIe] Op SIoIseo sap yueyodu0S 
saguLojsueN saguUOp sof Inod sameguIT uonefodsaqut | 
op uoNesI[NN | onb s[quiss [I ‘sopnya xnap so] suep JW. 
28P 19 WOU.) 2p surloel eT op iso Inb 99 Inod sinojyeurny 
-S9 «SINI[[IOW» So] QUUOP JUO CZ INdBIe] Bp NO YC INaSIE] op 
SIOISVD OP OpIe | B SOQULIOJSULI] SOQUUOP So] UOTas sagfod1e} 
-UI SOULIPSU Sot onbsIng ‘doURIJUOD op Sa][eAJOJUI Sop In} 
-IQANOD SUI[[9OXO OUN JUSIJJO CZ INVBIR] op NO f INasIE] op 
SIOISVD OP OPIv,] B SODUIOJSULI) SOQUUOP So] UOTas sagTod1a} 
-UI SOULIPOUI Sa] “SOpNia xnop So] SULP “OUIQU oq “JOISvO Np 
InosIe] e] sWOduT nad ‘aARIAI SNTIQuIS BI Op 19 JNeTAI sterq 
np so Inb 99 Inod soueLeA ev] op UONeUINSa.p saiatidoid 
SOINI|[IOU Sp JUDUIDIIOU SQN] JUSIJJO SOQULIOJsURT] SogUUOp 
Sd] UOTES sagfod19jUI SouURIPSU So] ‘sopnig xnap soy sued 
"%OT Op sgustoy9 
snjd sa] JOS saTeUIsLIO saguUOp S2T UOTAs saeg[odiajuI souRIp 
-9UI So] Inod Ina ,p xne} sa] ‘uone[ndod anne | ap syeijns 
-91 SO] SOAR OITULIOJUOD Ud ‘p UOIZaI e] op UONHe[ndod ze] suep 
gyeurumou Inayea ey anb syy0d snd yusWdATeOTIUsIS INdI 
-19 Pp xne} sap yusOduI09 sagjodiajul saguUOp sap souvIPSU 
SI SNOT *%O][ & sinousdns JUSWIIANKITFIUSIS JUOS INDO .p 
XN¥} SION] $39 ap xnap ‘O¢ INaZIE] ap SJaIsed say Inod ‘%O] B 
INILEdNs JUWISANLIIFIUSIS SO INDO .P xne} [Nas un ‘CZ 19 p 
INdZIL] IP SIBISVd Say Inod ‘stojaynoy, ‘a[euMUOU INIIVA PI B 
SINOLIQJUL JUOS SoINpgd0Id SION Sap DINUANOD op xNe} So ‘yp 


UOISAI P| Sp 9I[99 Jnes suoteindod sa] soino} sue “SegULIOJ 
-SUBI} SAQUUOP Sa] UOTES sog[odiajut soueIpeut say Inod qual 
-9JJIP USI 1S9 Ind .p xne} np [oid a] ‘Gueursrnas suoney 
-ndod xnep suep ayeurou Inge Py & sinatedns UsUIAANeO 
-IJIUSIS USI) B[eUTWIOU INoTeA Fe] & sINaLIgdns JUOS soyeUTSUO 
SQUUOP Sa] UOTEs sog{odsojUT SoURIPeU sap InJed ke sI[qeia 
OULIJUOD OP SOT[BAIOIUT So] INod ainyIIANOd ap xnv} S9y 19 
‘SII[NSAMUI ISO UOT[MHULYIE | 9p SULIPOUI LT SOAR DINIIANOD 
eI ‘sto} oun sJoouq ‘soddei3 ooave sgisop xnop e ued 
3] Suep sojadoi JUOS dINYaANOD ap sTIjod sao ap suTeLIAD 

“QIQI[NSIUI 4SO UOT[HULYIE | Op SULIPSUT LT DAv BINIOANOD 
BT ‘SJULSSOIDIUI SyeI[NSOI dp syI]IqGQIssod eI Isure JUesINpel 
‘SoT[RAIOJUI SOO B OpULIS JUOWI[TNUT InasIe] suUN oUUOp 
OULLIVA P| Op SUOTJEUII]SO Sap JIISOd sterq 9J ‘Oy “sojuepnid 
JUSWISWIQIIXE SINZ[VA Sop JUsOdUIOD sayeUIsIIO soguUOp 
S9| UOTOS Sog[Od19jUI soURIPSU Sop 10 L.A Sop ined kv stquis 
QOURIJUOD OP SOT[VAIOIUL SY] “%O[ op syeurwou sinajea 
Rl Op WSIOJJIp JUsWIONbNsNeIs Jso,U IndLe,.p xne} sao ap 
unone ‘jJajjo ua ‘amnauedns no seunuou onbsaid ainaanoo 
OUN JUdIJJO (SINdSIe[ $9jn0}) saguUOpP sap jUapuedap Inb 
sIatseo say ‘uone[ndod onbeyps sueq “aINEANOd ain] [ISU P] 
JUDTJJO SIQULIOJSULI) SAQUUOP Sa] UOTAS Seg[odiayut souRIpSUt 
sop 10 LA sop inied & si[qeia souerljUuo op saT[eAIoqUI 
sa] ‘soddei3 sues s1sap un e uejd np douasaid uq ‘asep 
-UOS op uryd 9] UOTAS JUOLIVA S]I ‘sowOJIUN Issne sed uOS 
OU OULIPSUI FE] Op UOTLUIT}SO pp aInpgooid onbeyo ep sourly 
-U09 OP Sd][VAIOIUI Sap sINLIOANOD ke] INod syei[Nsel SIT 


+08 1€0 80°0 %S°6 T€0 80°0 OSLI 
*%SL 0¢'0 60°0 ®v'6 1€0 KO) StL 
DOL T€0 oro %9'8 ZE'0 ITO VL 
be S 8¢0 870 *%T9 6€°0 670 OOOTOI 
#%TS Ir'0 Te0 *%e9 er'0 ZE0 000ZOI 4sonC, 
%9'8 6L'0 val) %6'8 860 6€°0 WS p uoIs9y 
*%6 11 870 Z0°0- MV 67-0 100 OSLI 
BEI L7‘0 10°0- %0'T1 870 Z0°0 Sal 
%8'O1 870 10°0 %0°11 670 70'0 VL 
DEL 8¢'0 L7‘0 *DL'9 or‘ 670 OOOTOI 
xDSL 6€'0 870 +%L'9 tr'0 1€0 000ZOI pus 
BV IL‘0 St‘0 bv Tl 88°0 0€'0 WS € UOIsaY 
~r‘OI a0) I1'0 %0°6 tr'0 st‘0 OS.LI 
Dr'Ol LEO Il‘0 %8'6 or'0 91°0 SLI 
4S 11 Ir‘0 vl'0 %V'Ol Iv'0 St'0 VABt 
%L'8 0¢‘0 LV‘0 *bOL cv'0 x0) OOOTOI 
%9'8 seo €7'0 +269 vr'0 €£'0 000ZOI ISOMPIJAI 
*%6L LOI Ir‘0 4%O'L vl Ls‘0 WS Zz uOIsay 
xT 870 ¢0'0 %S'6 970 S00 OSLI 
+%8 11 L7‘0 90°0 %6 01 S70 LOO StL 
HEU L7‘0 90°0 %0'01 alt) 90°0 vLi 
Eb €£0 610 *%0'L GAY 170 OOOTOI 
%0'°6 LEO cc0 *6 9 se0 S70 QOOZOL  jouuonUusAUOS 
SATIE[OI 
INI ,p xnep, sei Jue[al sietg == INoLa,p xney, es yueyal srerg * oe _ nies 
soddes3 sues 9139p un & Ue aInp2001g 


soddes3 dae sgisop xnop & ULI 


%6 Pp VoueIUOS ap sayfeAsoqut sap Inod InaL9,p XNe} So] 39 DOULLIVA | BP SUOTEUITSE SOP Inod sanejal syi[1qe}s P| 19 JNIAI sieiq aT] 
p nesqey 


sueipew aus ep xd sep sanbijdes sed BoUeLeA e| OP UONEWIISS,| 9 UONEWUNSS,] sueWBIS ja UOSdwoUL ZBL 


“SaQULIOJSURI] S9QUUOP SP] 
UOTaS Sag[odiajuI saueIPEUI Soy Inod JaIsed op sINeBIR] 
SIO] SB] DOAK DYOOI Js9 ay[IQuUIs eI ‘[e1oUes U_ “JneIoI 
sIviq np seo oj suep onb sgnbseur issne sed j10s ou 
gouara]JIp e] aonb uarq ‘sayeulsLIO SAQUUOP Se] UOTAS S99] 
-od19]UI SoueIPSU Sa] onb s}eI[NsaI SINIT[IOW op ‘aTeIOU 
-98 UOSe] op ‘QUUOP JUO SAQULIOJSURT] S9QUUOP Sa] UOTES 
sag[OdiajUl SoUvIPEUI Soy ‘SIOJ oUN s1ODU_ “UOTeIOdI9} 
-UL | OP IBeSSI] Op 19JJO,] Op asneo & nagud y1eI9 JeI[NSEI 
a9 ‘suonejndod axyenb sa] Inod a1901peul snqd ek] sytTIq 
-B)S B] JUSIPIJJO UOT[HULYDE | op SURIPSU e] Op 9oURLILA 
Rl OP SUOTJEUINSS SIT “SPI[IQuIs DINI[[IOW ve] JUSTeIJJO 
Sag[OCIOIUI SOULIPSU SAP SOURLIPA EP] Op SUOTIVUINSO SIT 


“(W'S UOT]DES BI IIOA) 
ynevY J SISA seq UN JayOdWIOd sagsoddns juOs soueLIeA 
RB] op suoNeUIsa soy onbsind ‘saddei3 usa sgisap xnap & 
uryd 9] Inod saprjos yuaWaTqenbsewlal UOS s}e][NSEI S9D, 
“UO][NULYDE | OP SOULIPSU Sap 19 SO[BUIZLIO SagUUOP SP] 
UOTas Sag[Od1a}UI SoULIPSU Sap JNeTAI sterq yWed snjd np 
d[[12}] Bl Op SNIOU! BP] Op SUIOW ajUaSaIdeI sagULIOJsue 
SaQUUOP Sa] UOTAS sagfodia}UI soURIPSU Sop UeOdUIT 
snjd 9] JHeyar stetq a] -no ‘suonerndod aynenb sap sion 


suep denbieul USI 1Sd UONVUII}SA,.P BPOYJOW! LI & dT] 
doualgIjIp ey] ned snid a] Jeped stetq aj JUaOdWI0D 


(OSLI ‘SZLI ‘pL]) seeuuojsuen saguuop sa] uoTes 
sog[odiajUt SoULIPSUL Sop SOULLIRA BI Op SUOTRUIT]SaSaT = — 


‘Jolpnonied uq ‘siy[iqeis eB] ap 39 Jeol 
steiq np jsa Inb 39 Inod sje}[Nsg1 sIne[[low say yUoUUOp 
SOSULIOJSURI] SaQUUOP So] UOTas sag[odisjUI souRIpsUI 
SOP DOURLIVA PB] SP SUONLUII}SA Sa] ‘sapnjg xnap so] sued 

‘%H(06 OP VoULIJUOD Op NRdAIU 9] Inod syeurWOU 
-Iq UOINQUIsIp eR] op speuoU UOTeUTxo1dde ,| op souesoud 
Ud QI ‘O Op [eUIWOU INIA .p xne} Np JUSIOJJIP JUSWIA[QeID 
-g1dde jso Inoa,p xne} a] aonb onbipur y neayqe} np suuoyT 
-09 dIQIWap | ap (,.) onbsgyse J “suOT[HURYIE YOO ¢ Ins 
JUSPUOJ 9S («9LI9A») OITTIQUIS B] 19 JINBTAI sTeIq a] Inod Ino} 
-PUIUOUAPp J] “SUOT[HURYIE COO | INs JUSPUOJ as oINIOANOD 
OP XN} S2] 19 SU]IGRIS PT 19 JHL[OI sTeIq 9] Inod sinoyeroUINU 
sx] ‘suoneindod anenb sap souelieA eB] Op SUOTICUIT}SS So] 
Inod uosteredwiod ap sainsoul SIO]) So] DUINSaI p NeITqQe} FT 
"SOJOUTSIP $9J} $JeI[NSII IP SOLIVS SION sUOTAR Snou ‘sanbutd 
-WId saguuop sap asAyeue a3j0U Inod sUIWIOD “s}UaIaJJIP UIIG 
919 UO s}e][Ns9I Soy ‘aInpgooid anbeyo Inod soueweA eI op 
UOTJeUITSS ,p sayaUdold sa] guIUeXxe sUOAR snoU onbsIOT 


SEV bro IL7P OtEL SSE 16¢ OSEL LIEV OSLI 

6£P £ €Ls 967 PEED 09r € Sle PIE V OLE V ScLI 

v6r € 66S CVE D €8eV vISe Sle COV v vIvy PL 

cese ESS Olvy Livy OES 8SC- 69¢ LLY Y OOOTOI 

L8V 6rr 6Lt v CET. BLS E S81 Icy py Str 000ZOI sono 
90S € 919 ISey v6 P 60S € OrI- C8E PV S8e Pr WS p uoIs9y 
0L6 7 958 089 € BLL Ee C167 SvL OTS 769 € | OSLI 

766 7 L8L WES C6L € V76 7 699 8E9 669 € SCLI 

870 € 99L she/he vI8e [S672 6£9 CONE SILE PL 

990 € CIT 08 € 908 € 176 7 101 OPLe COLE 000101 

Il0€ 999 6EL 96LE 866 C 6€S 699 € 80L € 0007ZOI pns 
PSO € 96L CSLE SEB Ee 167 10€ BS9€ OLOE WS € UOIsgy 
996 LAME SAE’, vTT 9 961 09l 8179 0% 9 OSLI 

IL6¥ C6LS ee VSL9 OLT9 LE6v L7I CLe9 €L79 SCLI 

6L6 1 806- 877 9 v67 9 ¥00 $ 87E c9E9 cLE 9 PL 

vOT $ VLEI- O08¢9 9¢%S 9 6£6 ESE L679 eve 9 OOOTOI 

620 $ [Loa “LOG9 cee9o 766 LoI- ¢L79 9LT79 000COI ISOMPI 
6S6 7 €SL- 877 9 €L79 9967 86S- L879 Sivas) WS Z uoIsgy 
9£9 7 CLE €8£ SOE E 909 7 6CE S9TE C87 E OSLI 

799 7 CCE IZEE LEGS LI9T 9LT C67 € POE € SCLI 

6IL7 ITE VIE E BLEE €L9T €L7 OVE E ISE€ PL 

PLL ZT 8L7- OcrE Ieve THO T pSt- B9E€ L8E€ OOOTOI 

$897 681 Ive € OPEC 869 @ 191 Olieae OEE OOOTOI = jauuonusauos 
CELT vce LEE 68E £ IL9¢ (4 SVE EL SPE E WS jWowsouRUly 

WOd.1 WOd.1 oueIpsut & 
WV srerg Ld reas WV sierg La” op omomy x tr Soe ter nda 
soddess uo sgi8ap xnap eR uo[[nuRYog seddeis3 ua uou 918ap un e UOT[HUeYIg aInpgd01g 


QUPIPEUI B[ 9p UOTIEUINSS p soinpgoold sa] Inod 
(INVA) 2uUsKow onjosge IndLa | 19 steIq 2 ‘(Lq) adA} Inova | ‘anburdwse PO,| op suloel eT 


L8t 


€ neaqeL 


000z e:quiadep ‘eyenbue,p senbiuyos 1 


9ILOI 8SEs LS8 
€v0 6 CCS €CL 
ces OL 997 S tvs 
SETI LSS 688 
Os SC v 


JOISed Op INIBIeT 


OZE IZ O€I Z9I (p UOIs9I) JsanE 

898 O81 Srl €€I (¢ UOIs91) png 

Lv9 O17 TLE IST (Z UoIsg3) IsOMpPII 

€9T TCT ELI LOI [PUUONUSAUOD JUSUISOUeULy 
6) ue Ips uonelndog 


SAQULIOJSULT] SO9|NUNIS SAQUUOP Sap B[EUISUO s[[aY99,[ Inod ssaIsed sap INasIe] P| 19 ofENb suUgIstoN a] ‘ouLIPSUI eT 
c HesyqeL 


“QUBIPOU B] Op UOT} 
-eUINSd .p sonpgoold sap SHI 2d} ap uonneda sear 

SOULLIRA | Op UOTeUINSS .p ss}gUdold sap uostereduiog «*Z 
‘uonje[ndod onbeys inod anne | e ainpgooid oun ,p 
sorese soid nod e juvj}9 souudAoU! sanfosqe sinoLe 
$9] 19 IIOF” Sop outro ey] ‘sanbnuapt said nad ve syeynsar 
Sop JUSUUOP sUPIPSU ek] Op UONeUITISS.p soinpgo0id 
XIS Soy ‘a}U9A op xd np 3so Inb 90 nod surow ny 
“snssop-19 aILI09p sonbuidura saguuop sop asdyeur | & 
JUSUISATILTOI SUOSANOS SOU SULITJUOD JUO s}eI[NS9I SoD 
“suoT]HUeYdE 
000 ¢ Op jtued e oginojeo 19 & sanbysie}s soo op 
sunoeyD ‘UoNe[NUIIs ap sopnye xnop sa] Inod oueipsul 
R] op uoNneWNse,p sinpgooid onbeyo ap (WVd) 
QUUDAOUL anfosge Ina9 | 39 sterq 9] “(L.q) dA} InoUe J 
‘onburdura JOT, | Op ouloel ev] ayuosaid ¢ nes[qe} a] 
ourIipoul 

R] ep uoNeUNse,p soinpesooid sop uosiereduiog ‘| 


syeynsoy 


‘HOI Op eS Inde .p xne} oJ 19 ‘O19Z op yUoIOYySoIdde 
9S SATILIAI SI]IqGuIs ve] nb uaiq Issnv JNeIAI stelq a] ‘ouRLIeA 
R] ap «jewindo» inayeunss un ,p souaseid ug ‘InoyeuTWIOUSp 
3] Suep WOA.I OP gies of asttin nb (267 “S861) JONOM 
SUeP SIUTJap SATVe[aI QUUSAOU onbyeIpenb now ,| sed 3so ,U 
SANLIAI SI[IquIs eT aonb Jojou YW *’4 soURLIeA ke] op UOTeU 
-1S9,] 9p “A‘D UN.p UOHeUTIxoidde oun so a][9 ‘souRLIeA 
R] Op SUOTILUITJSS Sap DoULIRA PI AP INSU OUN Sd ITTIQeIS 
ey] ‘ouuekow onbyeipenb Inova sfqeiLI9 ev] op uoTOdold 
SUIWIOD DOURLIVA RI OP UONRUII}SA.[ Op steiq np sInsow 
9UN 1S9 JILIOI SILI FT (9661) CLUS 19 OY SuEP 19 (8861) 
NM 19 ORY “IeAOY suep sasI[NN 919 WO SoIQILIO sap 


‘%06 
OP ddURIJUOD ap oTTeAIOIUI UN .p oInaLedns 9}1WIENxe | 1s9 9 


12 %06 
OP SOURIJUOS op d|[PAIOIUI UN ,p sINOLIgJUt gITWI9NXa,] Iso 9 


no 000 1/('6 
49 no Ig >) no suoyjnueyss,p sIQUION ANe110,p xn] 


I=f 
(Die l(000 T/e'QW-"4) “X)]_ aanepes angers 


1=f 
1 - (Qwioo0 1/4) oejas sterg 


SUBIPSW B}JUBA Bp XUd Sep senbijdas sed soUeUeA &| Bp UO!}EWNSA,| JO UONEWISE,] :uUeBWBIS je UOSdwOUL 


:yIns nb a9 & 
JUSUIDATILIOI SogIedUWIOD 939 JUO DOURLIVA B] Op SUOTeUINSO 
S9D ‘sUOTINULYIE QOD S SEP ONO I Jnod T suvIpeut ev] op uoN 
-eUINSo _p aInpgooid onbeypo e yuepuodsaL09 (4) SHW 0dA3 
OP JOURLIVA L] OP UOTVUINSS oUN g[Nd]Ted suOAk snoU ‘aueIpP 
-9UI B] OP UONCUINSE Pp SOPOUISW SOJUdIOJJIP Sop souLLIVA 
BR] 9p UoNeUIse.p soyoudoid say Joredwiod ep uly 
‘(117-607 “9861) 100159q ep UONTUIJEp ke] JUBAINS 


000 ¢ | 
Bias =, apk= Cu AVIN 


Eoeaes ieee 


SULIOJ B] SNOS T URIPOUL P| Op UOTJeUIT}SAa,p sINpado1d anbeys 
oP (AVW) suusdoul onjosqe InoLa | gnoyeo suOAe SNON] 
(0661) SuLypne suep a}I09p onbiuiduisa suuaAour 
onbneipenb inaid.| e[ 189, ‘uone[ndod ey op sueipow 
RT 89°59 19 “5S np ouusfow ve 3s9 «9 T Ino}eUINSO] 
32 I uoTMUeYDE,; Inod aguMse sueIpeul vj yso “) no 


(ZS) 


(1's) (2), s¥1q + (9),2 = 
000 $ 


— = (IN 
He) ae 


dope Ve 


QULIOJ P] 
Snos f 9UBIPEU P| Op UOTVUIT}S9,.p sINpgo0Id ev] op onbirdue 
guushoul onbyeipenb inae | ‘(9) A s[noyeo suOAe SNON, 
“SapULIOJsUe I} 
soguuop sa] Inod s[euUIsuUO s[eY99,[ op JoIsed op sino3 
-Ie] Soy onb aurgui ap ‘aruly uone[ndod onbeyo ap sjmsenb 
QUIZISION 9] 19 DULIPSU RP] aJUesgid Z Nes[qQe} FJ ‘(e;dwi0D 
ne uopMueyos,| ‘olez uonnede: ej sudutod A) suol[y 
-UbYd9 -IWap xne soanqije 919 1U0 SsguULIOJsURT] SoguUOp 
$00 18 a[duroo ne *G uolnueyos,| ap opie] e saquuojsuey 
919 1U0 SaTeUIZIIO sagUUOp SIT ‘OS LI 19 SZLI ‘p.LI seinpgo 
-oid saj Inod stoj oun noly no & aITeQUI] UONPULIOJsURY] BT 


(saguuop sop puadgp inb Jars ap Inesie]) 
OS e[le1 ap siaiseo “USWoIIegUI] SogULIOJsueT] 


sogUUOp Sa] UOTES seg[odisjUI soURIPSUI so] :0SLI 
(saguuop sop puadgp tnb Jorse9 ap Inasie]) 
SZ d[[le1 op sIoIsed “WUSWOIIVQUT] SegULIOJsUeT) 
saguUOp Se] UOTEs sag[odisjUI souRIPSUI so] “STZ7LI 
Ost 


0SZ OSs II Sz7sss‘0 
OOE OSs v1 6Z6SS‘0 
OSI 00S 9 SE8Ss‘O 
00S OSI SZ O€0LS‘0 
u N d 2 
uol[nueyos,|  uonejndod (aju9A ap xud ‘inayeoyyneNs) 
Op oe L Rl op apeL uone|pHOD 


778S‘0 = ISL Eg © enbrwyytesoy ayewou ysang 
6rSS0 }=—s bly 6@ «= SNDrWYLeSo] sfeuOU png 
LS6S‘0 = «LOS TE = eNbrWY eso] eTeuJOU ISOMPIAL 
: [ouUONUSAUO0D 
S68v'0 © SLE LZ onbyunUESo] ojeutOU Ls aoueuty 
fe) () uonnquisig uonetndog 


QjU9A op xud ap sanguivied 


salnens suoj[MUeYoe sap o][Te} 19 sag[NuIs suone[ndod sap sonbysusyoeseD 
[ neaquy, 


(saguuop sop puodgp inb Jatseo ap Inosse]) 
p [le] op sJoiseo ‘JudWOITe9UI] sogUOJsURT 
SSQUUOP Sd] UOTAaS sog[odJa}UI soURIPSU sd] ‘PL 


(OXxIf Jorseo 
OP INdZIe[) OOO I Ile} Op sJoIsed ‘sayeuIsLO 


SoQUUOP Sa] UOTES sag[odiaj}UI soURIPsw Sot :QOOTOI 
(OXxIj Jotseo 
Op INIZIR[) OOO Z AIf[le} Op ssdIsed ‘sayeuIsLO 

SOQUUOP Sa] UOTAS sag[odiajUI soURIPsUI So] :Q00ZOI 
uoT[HUeYS9-TWop 

enbeys Inod uol[nueyss,| op oueipou ey] “AIA 


‘yins nb 99 op UOT}OUOJ Ud sUOTTHUeYD9 
000 S sep sonburdurs (Wryq) souUsAOUI SanTosqe SINILIO So] 
19 (NOW) souusXhow sanbyeipenb sinawos soy SuITSS SUOAR 
snou ‘sgjedai suo][WUeYyde sop Inod suRIpsul ek] op UONeUI 
-Ns9,.p sinpgooid anbeyp ap uotstogid ey] JOUIWIeXS .p UY 
"QINUTISIP OpNy9 ouN Joy 
-nsnf Inod suoneuwinse soy suep yuouruestyjns sed yua}dwioo 
OU DOS BX ap ueyd np sinajdes sane xnap saj ja ‘snsusa 
oy) Jo neaing np saygnbus sayne.p suep gsi[yn juourel 
-B1 1S9 ‘JOS P| Suep s[QeasI[39U UOU aonb UaIq ‘soIsap SION 
B UoT[HURYSIT “(DOS P| ep UoT[MURYDg,| op UOITAUS %C) 
dd Sep dN UolHuRYys9 | NO “(DOS PB] ep uoT[MURYD9,| 9p 
UOIIAUS %Z) snid no jUsWIaZO] ap saytun burs inod stwued 
Sep dN uoyHueyse, | “(OOS BI ep uoT[MULYD9,| ep said 
nod & %¢Z ‘sei8ap SION eB UOT[HURYD9) sJreUESgIdeI0jNe 
uou stuLiod ap xnvainq sap jo saanejuasgidasoyne uoU Fg 
SOP SULP SUIOU NO JUSWI9ZO] op sayuN oyenb nod stud 
SOP ANS uoT[Hueysg | Jopnuris op 9]u9) sed suoAe U SNON 


"jney J] SIA SIeIq UN $9yNo} USUI 
-9[qeqoid jusyodu0d soueLeA B| ap SUOTeUINSA Say aonb 
9UOS Op ‘(ZZ ‘S861) JONOM SPURUIWIODEI dT SWIWUOD 
UODEI9S BP] Ip gIsop Jomueid ne sseuUOT|NULYDE,p 
suonoedy SojueyOdul _p a}dwioo sed juspual ou sayedel 
splod sa] ‘ayes SUIQUI P| Op INSLIgIUI | e gIZap JonuaId 
np (qdN) eseuuol[nueyse.p soyun xne jodde yey 
SHW 2d) op uonnedal eT ‘gisap Jared np sajtun sap 
quawiednols 9] sed puaiduiod au 12 OOS BI Op Ldd eseu 
-UOT[NUeYD9,.| 9p o3dWOd sed yuaT) ou apNyd 91199 ‘aTTIe) 


621 


OUIQUI B] JUO Yd Sse sono; onbsing “qq sep insrd 
-91UI | & Soylun .p onbryeuraysXs UoT[HURYOE UN Iso gIdEp 
suieixnap xq] *(¢ = 4A) oiens sed qqf xnep ap aster 
SUBS dIIOJegIe ses UOT[WURYSE UN jso gIZEp JOTI 
-oid a] ‘agjnumts uonendod anbeyo inod saisap xnop 
@ SUOT[HULYIE YOO S$ 211) SUOAR SnoU ‘apnye 97309 suUeq 
(DOS BI ep uorHUeYsg,] op seid ned e %Op) soanea 
-uasgidoiojne uouU Ad Sep suep sjiyejUasgidos0jNe 
stutied ap xneaing sap 39 saatjejuasaidajoyne Fqn 
Sap suep sjiejuassidosojne uoU stuLied op xnvoing 
Sap SUIOU! NO JUSWI9SO] op sayuN aNenb nod stuwu4ed 
Sop qNS uo][Hueyse | ap aiidsut.s uejd oweixnop sy — 


“sjoued xnoap ud uoT[NUeYdg | JUOW 
-anbiyeuigisAs 9sIAIp SUOAR SNOU Jd SJULTJNeNS B[QeLIeA 
RI UuOTaS ayes onbeyd sp INsLUIg\WUI,] B UOT[MULYDY | 
9) SUOAR SNoU ‘SH dA} ap UONNedeal ouN Jon}99jJO .p 
ulJY ‘ayers onbeyod suep aseuuo][NUeYSE ,p xne} OWIQUI 
Np opie | & sgjnwis uoNe[ndod onbeyps snod astral sues 
SarlOjegTe SAIJNENS SUOT[HURYIE OOO S 914) SUOAR SNOU 
‘apmia a7199 sueq ‘(DOS BI ep uoT[HUeYDE,| ap said nod 
B YZ) saatyejuaseidosoyne Yq sep suep stunied op 
XNvIING Sap SUIOW NO JUIWIIZO] ap sayuN ayenb inod 
stuuied sop qs uoynueysg | op culdsur,s Jommeid ay — 


:syUdTajIp oygnb 
-ua.p suejd xnap inod suoj[nueyse ,p Sotigs xnap 911) SUODAR 
snou sind ‘uonetndod anbeyps suep ojese aye) ap saves OS 
QULIOJ SUOAR SNOU ‘saTUTZ sUOnE[Ndod sa] 9919 OAR saIdy 
"| neg[qe} a] suep sajuasgid jUOsS 
(u) suoT[HURYDE Sop a]IIe} eB] 19 (VV) UONR[Ndod ex] sp aIIIe) 
Ry ‘(aIURIJIeINS BIQeLIeA LJ Jd S9[NUWIIS 9]U9A Op xLid say anu) 
UOT[HULYIS | Op SUONLIIOD Soy ‘ajapoul ap saneuesed say] 
“UOT WURYDS | op suoTye[ndod sap ined ev ouINsa AY 9] 1IeI9 
uonetndod anbeyp ap aye) BT “(.6 1) ZoIMe|S] 19 UN[SLOPY 
SUPP S}LIOVP SOINILI9IXS SOINIQIO Sap ajsnqol 9[3aI ke] 
op apie] & soguuddns juarej9 sajueuoge sinayea sat onbs1o] 
jnouns ‘ayeursi0 uoNendod ve] ap ajuepuodsaui0s onbn 
-Stye}s Be] op dnooneagq juareyooiddel as sgjnuts uoNne;ndod 
onbeyd op 93U9A op xud np a][qQeLIeA ek] Op UOT[HULYDE,| 9p 
quowessnede | 19 UOT[NURYSE,| Op sLNaWIASe | ‘SaTUDdIEd 
SY] “9U9A Op xd np s[qQeUPA ke] 19910 & JURAIOS PIepUrIs 


0002 aiquiecap ‘ajanbuse,p sanbiuyoe | 


Q[LULIOU aIOyegTe a[quuIeA eI x 9‘Q B QUUOTIPpe jULId 
sua} 99 *g‘Q Jed sordnynui ‘prepurys aitojyegye syeuOU 
IngjeA sWgIxnep suN,p sjuepusdgpur uonee1o vy sed 
ONUs}qOo 9}9 B a[[9 ‘o[BUISIeW prepur}s s[eULIOU UOTNGLsIp 
aUN WeYOduoOd s[qelIeVA 99D “gI8ep Jotureid op soddeids 
SOP 32 SI}VIS SOP JOULIOJ & IAJOS & JIQULIVA oWIaIXNap 
eT (9) spsudoidde juoweoejdwia.p sonouiesed ap apie] 
R ase[eogp UN,p 30 souRSsInd eT] & UOTeAQTS OUN,p STAINS 
‘Sssop-I9 SQUTULIO}9p SaTjgueIed sap ape | F -0 SULLIVA 39 
} SUUSAOUL SOAR SITOJeQTL J[PULIOU [QUA oUN.p UOTIeSIO 
R] Jed onusjgo 919 & d][9 39 ‘SONPUDA SUOSTeUI Sap aJUdA Op 
xud 9] yeyUssgide1 uoneindod onbeyo ap sayqeuea xnop sap 
srgnueld BT (66 ‘8961) NYD 19 Yorping ‘Apureg ‘Jo[AeN 
SUEP d}LID9P SpoyoW eI Op opie, & 9‘Q=d ap onagid 
UONe[S09 oun jURAR Sa[qGeLUIeA XNop & SdIUT} SagTNUIIs suoN 
-e[ndod anenb 9919 suoAe Snou ‘sa[apour SoU ap apre | V 
‘QULIOJ ap aneutesed 9] 189 9 19 
J[2Y99 ,.p oNQWIeIed J] IS9 9 ‘[Inas op oNeWIeIed 9g] Isa g NO 


co>d>Q Inod 


9 Z SEG ipa 
(Ae =@-aygop PO OF 


Susuop op UOTDUOJ e] aUOduIOS snbruyLesOo] 
Q[PULIOU UOTNGLYSIp eT ‘giUN a[Nas ouN ev SaNpUdA sUOSTeU 
sop 9]U90A op xud np uonnquisip ek, ap uoneunxoidde 
oun J1U9}q0 ,p stuied yuo sonbrumyieso] soyewOU suOTING 
-I)SIp soq ‘TI aun] e g100sse UOT[WUeYDE,p sprod 92] 1s9 
‘mM NO Taseuuo]nueyse .p s1tun anbeyp nod sjusuransiZarue 
'M 9919 SUOAR SNOU ‘SaNpUdA SUOSTeU Sap 9]U9A op xUd np 
aruly uone[ndod ve] op uonewrxoidde oun 31u9}g0 .p UY 
‘DOS PI ap suorsa1 
sap suoieindod sap ajuepuadepur sed 3sa,u QOS BI oP 
jususoueUly op ad} np uoTe[ndod x] anb Jaj0u Y ‘suoTeUr 
-1]S9 sap allnpoad B yURAIES DOS ke] op suonetndod sap sadAy 
SIQAIP JUaIeJUasaideal suonendod sad ‘(Pp UOIseI = sONnE) 
32 € UOIZI = png ‘7 UOIZaI = ISOMPIJA)) SO[CUOISEI SUONL] 
-ndod ston 39 (JeUUOTUSAUOD JUSUTSOURUTJ) JUSWIOURUTY 
op odAj np uonejndod sun :D0§ ze] ep UOT[NURYDg,| Op 
suone[ndod anenb ap seauuop sap asATeur oun Ins ssgpuoy 
SOIUTJ SaT[atoIyIIe suonetndod aNenb 9919 suOAR SHON 


UOIE[NUIIS 9p Sapnyza say anod ainpe3s01g = °V 


NOILV'TOWIS 
ad SAGNLA SAG NOSIVUVdNOD ‘S 


“QUPIPOU P| 
ap SINdJeUINSE s]UdIEIJIP Sop spre | e searedaid GHW eA) 
ap soULLILA ¥] ap sUOTeUINSS sap solatidoid say Jon[eag.p 
ue oOeD ao; ep UOoHL[NWIS op spnig oun gQUOUI 
SUOAR SNoU ‘sjUBN[OUOD sed juarei9,u sonbisiduia syei[nses 
sou onbsing ‘JaInsoul so] op yuRNoULIEd «Io-uOTe}9» UNNL 
J0 DOURLIVA PI OP SUOTILLITSS ,_P SOJOUNSIP SALIQS SIOI] SUOTAL 
SNON ‘saxa|diad sassie] yUO snou sonburduts syey[Nsol SOT 


‘sagjede 
Ld se] anb sayned snjd juoursyergues juateis JynNIpPOO MA Op 
LY Se JuoursyseIIp agfnoyeo (4) 1.q| & [odde yuorestey uon 
-onpoid ap sulgisAs np JJNIPOOM ap L_ sey anbsso7T -suon 
-nedai sap JUSUIA}DEIIp 9)1e.1)x9 Isa Inox | anbstnd ‘sanbryd 
-RISOUISP SOINJIONNS SOSIOAIP OP SIA-R-SIA saysnqor snd 
Os (SIS) WA sepoulour say aonb npouos yuo sj 39 ‘ayned 
dow) JUOANOS 3S9 JJNIPOOMA OP 1.7,] ‘sevd 99 sueq “(DOS 
R] op urd a] SUID) sagTaLION JUAWIALIO] sagredas so[qeLea 
Sop INs JUSPUOJ 9S sales so] NO ‘saljHeNs suo;[HUeYyIE 
Sop Sed a] SUP Jnes ‘sa[qe[quias sojatIdo1d sap juaIeAe 
Sepoyjsul xnop seq onb 9anod} ywUO SII 10 “WY_ epoyyour 
RP] SP Sal[99 19 JJMIPOOM OP LA Sa ereduros wo (gg61) 
NA 19 OY “TRACY “osI[WN ssvuuo][YUeYDE.p ued say Inod 
sogtidoiddeur suroul np NO Sa9UII}S9-sNOs JUOS JINIPOO AA 9p 
IPOS LI op sogtqnd [q sey onb jesayquias Ir ‘snd aq 
‘sagttqnd saguuop sep 9yTu 
-Jojun year A [I,.nb ainjouoo sed yreanod ou uo | anb ayos 
op ‘sogitqnd sadA} sinolia xne yuR[quiessal sad) sinoLa 
Sop QUUOP &.U SULIPSU! ke] Op UONeUINSA p sapoyieu sep 
ouNoNY “UONR[OdI9}UI ,p JaJJo UN Jd SITeSUT] UONeULIOJsUeN 
oun year A [I ‘asouapIAg 33N0} aq ‘saeyooiddes sa sojno} 
WsIe}d LA Soy “(SAsULIOJsUeI) NO Sa[eUIZIIO) soguUOp ap 
ody ourgul un Inod aya ug ‘sdaTodia}uI soueIpeU op Sates 
xnop sj Inod [.7,] ap SuoNeUIss soy 39 JOIseO Np InosIe] 
RI oud JOU UdT] op sed year A.u [J] ‘(ssquUOp sep puadap 
Inb JoIsed op IMosIe]) SopULIOJsueI) SeQUUOP Sa] UOTES 
sag odiojul souvIpeul soy Inod alias aun 49 (AxIJ JOIsed ap 
INIBIL]) SOTLUISLIO SagUUOP Sa] UOTAS Sdg[odsa}UT SoURIPSUI 
Sa] Inod ags ouN ‘UOT[HURYE,| ap oURIPsU eT Inod dLIVs 
QUN :1.,P SO}OUNISIP Salas SION eB SIOJO}NO} Not] JWeUUOp 
‘SaTqe|qUUOS Sai] SUONeWINSa sop iINpold yUO JULIPSUI e] Op 
UONCUINSE Pp SOpOyJoU XIS Sa] aonb 9AJISqO SUOAR SNON 
‘suonniedal 
OOZ SP aple,] & ayInpoid 939 e DoURLIvA PI Op SUOTILUIN}SO /p 
auigs onbeyD ‘souvipaul ap uoNUI}Q0 | ap ona Us segdnois 
yueyg soygnbua xnep sep seguuop se “(qN uol[NueYysg | 
Inod sgsteiq uou 19 qQs UoT[HUeYD9,] Inod woLsod 
e solynens) DOS ke] ep UoNonpold ex] op uoNeUINsa.p 
yO QOULLIVA ap soulasAs say Inod onb sprod souwlgui soy 
QSI[TIN SUOAR SNON “WWousDUeUTy op adA} 9] UOTES Jo UOIS9I 
BR] UOTas sauvIpsUT sap gredgid suOAR SNON *L661 JOTTENE 
19 L661 unl ‘1 66| leu! ‘L66] SJeUL :sonpUdA SUOSIeU Sap 
3]U9A op xUd 9] Inod sueIpsul ve] op UOTJeUIT}SO .p SopoyyguUt 
Sop SOULLIVA PB] JOUNULX9 Pp Ue DOS P| ap uoT[HULYSE | op 
saguuop Sep stow anenb gstjgn suoAR Snou “UaUATeNIUT 


SANOMIdNaA SAANNOG SAG SLVLTASAU “FP 


‘JOS arenbua | op saysATeue so] soide .p 
gssed a] suep asnomonyy sutowl no snjd i9 & atso[opoy} 
-9UI aed ‘UoToeNSNOs Jed SgUIT|Se SIO[e SO OURIPSUL P| OP 
LAT ‘(uonnedes op sopoysul ap opre,| & epuiNse ang ined 
(D1, [) ouerpau ey nod %98‘7G ap soueljuod ap a]feArajut 
UN,P AANILEJUL OUI] Bf] IWINOy e UOoSey ap soouanbgsy 
sop sAe[NUIND UONNGIIsIp ey ns (4) La + G) offearoqut | 


SUBIPEW O]UBA Ep xUd Sep Senbijded sed BOUBLIEA B| BP UONEWUSA,| Jo UONEWISS,] sueWBIS ja UOSdwoUL BLL 


ap uonosfoid dace (QURIPsU e] Op UOTeWINse oUN Inod 
0S‘0 = d) d uoniodoid oun ,p aanewnysa odAj Anaue | e jedde 
WJ JJNIPOOAA op opoyjsw ey] ‘ouvipsw oun .p odAj Ino | 
Janyeag,p ule (Z7C61 JJMUPOOAA) JJMIPOOAA 9P spoyioul 
ey @ jodde ye} DOS eI Op UOHONpold op ougshs JT 

(8661) uosdwoyy 
suep suonnedel sop uonnquye,, ins soiejuswgtddns 
s[iviap sep eIoANON UG ‘(sajUepuadepul sogsoddns juOs 
DIES RP] OP AdN xXnep soy ‘uo[MUeYdI-1Wap DAe O[VUISLIO 
SpoyIou BI SUBP) SagTP1IOD JUOS sjouRd xnap sap se}TUN sap 
onbsind s0ueLivA PB] ap $9}901I09 SUOTIRUIT|SS Sap ITUINO; sed 
au ap onbsti apoyyeul 91199 ‘DOS BI op asepuos op ueyd 9] 
Inod a]quodsip arZayens a[Nas eB] JUSWIAT[aNUASSO }10s 39 onb 
uaIg ‘(IE ‘S86I1) JAOAA O1938Ns Z| SUTUIOD sjoued xnop 
Ud JUUOPIO UOT[HULYSE | JUaWIaNbHeUIS}sAs SUOSIAIP SNOU 39 
‘uONRSLIOINe .p ayep eT 39 onbiydess09g3 apod 9] UOTES SOANP} 
-uasgidaiojne s9yiun ud spuTUajopoid oseuUO][NULYDE ,p So} 
-1UN .p Sodnoj3 op INsLIUL | B STULIAd say SUOL) snoU ‘SHI 
odA} op uonnedas oun Inog ‘doURLIvA eB] Op UOTRWINSS.p 
ajurjystxogid ainjonys ey op onbiso] yuswWasuojold un y1e19 
wag od} op uoneordde oun ‘auedde soualayjip op inoyew 
-19S9 Ing] Inod saaneiuaseidesoyne uoU saze1s sap sdnols 
elap uareae DOS BI ap saistsojopoyjou say onbsing “(0661 
Alan’]) DOULLIVA B] OP SOULIOJIUN SUOTRUIIISS Sop alINpold 
op ule jnejuasgidosoyne uoT[MUeYyos uN Inod jyewtxo1d 
-de oseuuo][Hueyoe p o[NWIOJ ap InayeUIT}sa UN zB 39 JINR} 
-ussgidazoyne uoU UOT[MUeYSE UN Inod (gIedde sousIajJIp 
Op INdJeWINsS2) ZJASY op Inayeunse un ve jadde yweJ QOS 
R[ op uoNnonpold ve] Inod souvLeA ap [anjoe oulaisks FT] 


"SOATye]UDSoIdatOyNe UOU sseuUOTI[N 
-ueYd9 Pp sain soy Inod sagdnoi3 sayeNs ap INdU9IUL,| 
® (qd) g1sep Jord ap esdeuuo][HUeYse ,p soyUN soy 
USIOS s{I Nb 39 ‘saatejUasaidasoyne oseUUOT[NURYDE ,p 
SoUUN soy Inod sseuuo][NueYdE .p SayUN Pp INdLI9IUT,| 
B sjoued sap JUSIOS SUOT[NULY9-IWIAp Sap SJUIWIIIP Sd] 
onb uodsej op uoT[HUeYyoe-IWep np aisaj}eys ej Jonbydde — 


J9 S«SO[Qe[QUISS» SOATIE) 
-ussgidaiojne uoU sseUUOT[NULYOE ,p S9qIUN (STO) NO) 
xnop op juowaedde sed sagdnoid sayens sop IoWWOJ = — 


‘Q[VUISLIO VSeUUOTNULYDY,p SPpOyoU P| Op apre | 
B aseuUOT[NUeYSE ,.p oyun Jed sjoued xnap ud SdArjR} 
-Ussgidaloyne aseUuUOT[NULYIE,P SOUUN So] «JOSIAIP» — 


eR 
9ISISUOD o}eNS Jed aseUUOT[HUeYIE ,p 9}1UN UN Jed g}IOSNsS 
sWIg[qold 2] Jopioge p ajUvINOD UOdSeJ oUL) ‘sIBOP sUIAI 
-xnop ne sguUONdetas 919 & 9]e.NS Jed 9}1UN a[NEs OUN JUDA 
-Nos jo ‘sonbnewigysks suUO][URYDE Sap JUOS SoIZOp OUIVIS 
-10J] Jo OWIQIXNap sey] ‘ayes Jed qq~ oun suUOTdeT9s UO 
‘9130p Jonuaid ny ‘ayes red aseuuo][NueYyoe ,p soitun xnop 
B oSepuos op uryd un jso,U GN uel gj IU GAs ued J] IN 
(0661) SUFAPNE OA *,(5‘0-1)/T OP 
Ina\oej UN.p s[qrey doy yso sagyadal suoNeUINsd Sop IIOA.| 
anb sored inayerguinu 9] suep (fp) anenb alyyiyo a] sWOdu09 
uOIssaldxa seg ‘jJo[duIOD UOT[HUeYDE,, op oULIPoUT 


ZLt 


eB] op uoNeUNss,| oUsISsp O edIpul.| 19 (y*‘Z‘1=4) 
sajedai I suvIPSU PB] Op UONRUII}SO,| DUBISEP I SdIpUT,]| NO 


(€°Z) = (Peas 


[Sepp f 
{CPA - PAIN) <*> 
Y 


Jed aguuop 3s9 SHI 2dA} op uoNNedes oun Ins sexe 
QUPIPU B] Op UONeWINSo UN _p odAq INdIA TJ ‘0 19 Z OP Nal] 
ne CQ ap 39 ST op uoNNedas ap sprod sap e [adde yey SHIN 
adj} op uonnedal ey] ‘prewepey ap SooLeul sap Ins sexe 
uonnedai op sinpgodoid ey] ap sei[IeJap UoNduosap oun (C] | 
“PILI ‘S861) JONOM SUPP PIDANON UC ‘preWepeH op souyeul 
9UN,P eple.| & Segstogud JUOs (sUOT[HULYD9 -IWap) suoNnedas 
SY] ‘O19Z UOeIgpuod ouN jUO seaT[a aNUa,p 9IIOW eI IS 
ouigu UOINedgi onbeyo suep sastiduros juos Yq enbeys 
Inod suoneulyss sop ‘Isury ‘(soyun xnop soz ajuasgides 
a[o,nb uodsej op) Z red aguuonoseyas 9}1UN | op UoNeIJgpuod 
Jed ja asied onbeyo ap sytun oun .p uonoeyas red sgnqdajJo 
1So UOT[HURYDE -IWap deAv UONNEdaI oUN “WY epoujsw 
R] UOTAS ‘ayeNs ed YqQ xnep op ssepuos op uvyd un inod 
anduod doURLvA Bl OP UONPWINSd .p SpoysU ouN so (UOT 
~uBYo9 -TWap dear eguqyinbs) yu Uonnedel eT (ZST-OLT 
‘SR61) JOI0AA Jed ayLID9p UOT[MULYD9E-TWIAp DaAe DOURLICA 
Rl Op «a][OUUOTIpeN» soiqijinbg UoHeUIsS | op UONeLIeA 
aun 3s9 SH edA} ap uonnedas eT (SHI ed} ap uonnedgr 
e] Inod (Q661) surypne ‘eguqiinbe uonnede: ey snod 
(L661) BUNZ 19 DABIVAOY *(9661) OBYS 19 OLY *(Z66T) ENA 
32 NAA ‘Oey ‘ajdurexe Jed) sat[qnd uoNe}UoWINSOp e| suep 
guuotjuoU onb [2} ouURIPSU oUN ,p doURLIA Pk] JOWINSS Inod 
(0661 SUyPNE 39 G86 AeA) (UOTTHULYDg-TWIAp IeAR I9IJ 
-IPOUW!) SHIN Uonnedal op spoyyeu P| ast[WN sUOAR SNON 


JULLIVA LE] 8p UOHVUISY “gq 


"SIaISVd Sane so] onb oBIe] snjd UIWIOD SI9PISUOD j$9 [I 39 
‘saguuop sap %¢Z B nbsnl yUaTUOD JoIsed JOIWap 9] ‘UONIU 
-IJap sed ‘Isury “(WINWIxeUl & OOO 1) op SinofNo} Iso SOQULIOJ 
-SURI] S9QUUOP ap JUSWasse]d JoTWEp a] onb ays ap ‘C00 I 
B JINILI9JUI SINO[NO} jSo SOQUIOJSURT] S2QUUOP Sop oUPIPSUI 
v1] *O ve sainaLigjul saguuop sa nod sJatseo sap INasIRJ PY 
OP SIIOU 9p UOTONped L] Ip 39 SJOISed Op SIQUIOU Np USUI 
-3]qnop np 39JJ9,| Jourwexo Inod ¢Z aI[Ie} ap JoIsed 9] IsIOYD 
suoAe snou ‘*G ze sanaLgJul ssguUOPp SzJ INOd syaIsed QZ Op 
[210} UN IIOAR ,p UlJe QS SII) Op JaISed d] IsIOYD SUOAR SnOU 
LVIVIGIE JUSWIIANLIAI 919 & OS 19 SZ SINIZIL] Sop UOT}D9] 
-9s eT ‘GS ke sainouigdns saguuop sa] soyno} JURUa}UOD puss 
snjd sorseo un ya ‘*G ke sommaLigyul soguULIOJsueN saguUOp 
Sg] Inod p a]]Ie} ap sIsIsed QCGZ kB A [J “SoIsed op sIquioU 
np 1sa nb 39 Inod QOO Z aI!) Sp SISISeD xn soTqe{quas sts 
“TOYO 919 UO Ff dI[!B1 BP SIOISeD ST “OC 19 CZ “p AT[T21 ap Sols 
-B9 LIOISED OP SINIBIL] SP SI|QUIISUD STON B SINODAI NI SUODAR 
SNou ‘(SagWLIOJsuLI] SAQUUOP Sep UOTaS sag[od19juI souRIpP 
-9U1) UoNe[odiajuUI pp UoNdoO sUIgIXNap eI Jon[eAg,.p UIFY 
‘uonerfodiajul | op 99499, 
yesoureyus inb 39 ‘¢QOQO OSS ep eydoidde1 os sonpudA 
SUOSICUI Sap ULIPSUI 9]U9A ap XxLId 9] ‘SUOIZ9I SoUTe}IIO NO 
[etoads JuaWIDULUT ap SeLIOS9}e9 sop Inod ‘anb afqeAsou09 


000z esquiaogp ‘ajeanbua,p senbiuyoe | 


3S9 [J ‘UONLTJUL,| Op [ANIUDAD JaJJO ,| ISO SIOISBD S90 Op JUAUIIO 
-e[ due | & ot] JUAPIAY sUIgTqQold UP) ‘UOT|HULYDE,| op UOTBA 
-I9sqo opueis snid el ap puodap ines] ev] UOp JaIseo uN 
39 $000 9E SII} Op JoIseo un snyid “¢Q0O I aT[Ie} ep ssoIsevo 
PIS NO $000 Z IIe} Op sdaIseo /¢Z QUUOp PI99 :(UNUITXeUL B 
000 OSS) ep sinofno} jue}9 saTeUISLIO SogUUOp op JUSUASsE[O 
JOIUOp 2] ‘(666 SE & UNUUTUIW) op sIno[no} SUOp so soyeUIs 
-L10 saguUop op JUOWassP[S Jord 9] {FO00 OSS B INoL9yUI 
39 $000 9€ B InaLI9dns sinofno} jeJes ULIPsU 9]U9A op xUd 
at onb gsoddns suoae snou ‘stup-sye1q Sop sfquiasus | Inod 
a1uaA ap xud np suoneumsa,p stow sinarsnyd gurutexe ITOAR 
saidy [‘a[qIsstupe [IeJUSA9,| Wessedap so0sazv9 ap 31q 
-wou 9] onbsind ¢Q00 | & SINSTISJUI JOIsed op oT]Te} ouNONe 
youLIod OU DOURLIA PI OP UOTLUINSS.P XTdA [S1DISOT 2] 
BION] “$000 I A112) ap SAdIsed 39 (UONONpodd vy Inod soueL 
-CA P| Op UOTLUIT}SS .p [anjoe sUIQ}SAs 9] INod sgsT[N JaIseo 
OP INIBIe] DUIQUI BT) $OOO Z SI[!e1 Op SJOISed :(JUOUOSSE[D 
op sate}) JoIseo op sinasie] op soafquissua xnop 
9ST[QN SUOAR SNoU ‘(saTeUIsLIO saguUOPp Sap UOTaS seg[od19\UI 
souvIpoul) uoneodiajut .p uondo sigiwiold ey Jon[eAg .p UY 


($000 p = JoIseo 
ap InessJe[) WHA XI ep sigid sop queuusAoU sonpudsa 


SUOSIVUI Sop 9]U9A op xuUd np seuIsu0 UONNQISIG:¢ aaNsIy 


oo 
ee 
ee 
ee 
2s 
tt 


neocoe 
aveceoo 
AeRoes 
Haeoos 
naqveoo 
naecos 
waecoo 


“$OST 9 = S9ULIOJsUeT} 
uou 9][9Y499,, SUeP opueg op InosIeT (OCG = JoIseo 
ap InesJeT) WHA XI ep sigid sop ueuushow sonpusA 
SUOSIeU Sop 9]U9A op xUd np s9ULIOJsuRN) UONNQIISIG *p a4NsIy 


° 
oor 
oos 
oos 
oor 
cos 
coe 
oon 
oos 
oos 
oot 
oom 
oout 
eost 
corr 


"SOXIJ SIOISVD Sap B 
agloosse 3s Inb a][99 B JUSUIDATIeIedUIOD S[GULIeA SUTOUI 1S9 
saguuop Sop juepusdep JaIsed op SUOT[NULYDE Sep aI]Ie} BT 
‘snjd oq ‘soguuop sap ye99 | JURAINS 9}N0d snyd no spuLs3 
snjd ong ynod sguOJsueN UOU sT[oYO9,] SUeP JoIseO op 
[ITI BT “JUSWIIeSUI] SsgULIOJsueT] S9QUUOP Sep Inod s9i10q 
-R[9 SIOISVD Sop osso[dnos ke] JUSUOU p & | SOINSIJ say 
“Q[VUISLIO SOULIOJSULI) UOU I[OY499, SUP $OOS 
op satjeurxoldde inoBie] oun JusTeINe p INdBIE] ap SAQULIOJ 
-SURI] SOQUUOP Sap SIOISVD Soy 32 ‘aTBUIZLIO d][OYO9,] SUBP 
$OSTZ 9 INdBIR] Bp SIOISed Sap B JUOpUOdsaLIOD (¢ INESIL] Op 
SQULIOJSURN] SIOISeD Say ‘ayduraxe 190 SUL] ‘QS OP JoISvO op 
INIBILT SUN JOAR JUDWIIITVOUT] DOULIOJsuRI] UONNQMSIp eT op 
sUILUeISO}SIY | aJUaSoId p aIN3Yy ey] **G snod $966 HZI B UO 
19 “$087 801 @P 1S9 UONNGMSIp 91190 9p suRIPsU BT “(UOTIE} 
-uosgid ap SuOSIeI Sap INnod SISIOYD 1S9 JaISed Op INIZIV] PT 
‘SIOJ DUN BIODUD) $QOO F OP JoIsed op INasIE] oUN DaAR “WHA 
RI Op sigid sap JURUUDAOUI SONPUDA SUOSIeU Sap s[eUISIIO 
UOTINGISIp Bl Op SWUUIeISOJSIY UN dJUaSgId ¢ OINBI} eT 


“$OST [] = 99ULIOJsUeT] UOU ZT]9Y99,| OP OG = JoIseo 
op InasIe] ouN e JURpUOdsaOD IaIseD Op Indsse] oun 
Inod JauUUONUSAUOD JUSUIZOURUY UN yURUUaAOUI SoNpUdA 
SUOSTEU Sop 9JU9A Op xUd np sguLIOJsueI] UONNQUIsIC :7 aANsIy 


“($000 SZ = JOIsed Op INosIe]) JUSWSOULUYy 
9p JOISeD Op B[[OUUONUSAUOD InoSIe] oUN JURAR SaNpUdA 
SUOSIeU Sop 9]U9A op xd np s[eUIsu0 UONNQM)sIq:] a4nsiy 


SUBIPEW OJUBA ap XUd sep Senbijdei Jed BoUeUeA B| BP UONELUI}SA,| }9 UONEWNSS,] :uewBbis ye uosdwoUL OLI 


AC 
QINBI V] IIOA) JOIWIap do] JNes sIaIsed so] sno} Inod souLIOJ 
-tun snjd dnooneaq ju0s saguuop sap Juspuedgp Inb siorseo 
Sop INILIQIUL | B UOT[HULYE,p SaT[Ie} soy ‘UONdsoU0d Ine] Jed 
ap ‘anuod eg ‘(| dNBLJ &] MOA) JoIsed op UoT[HURYDE ,_p so[q 
-CLIVA ZOSSE SOT[Ie} SOP SUIL.IJUS SOTPUISLIO SOQUUOP So] DOAR 
SOXI] JOISeO Op sINeZIe] op UOHesI[NN,] ‘sopoyjyoul xnop 
So] Inod anne | & Jaised UN .p UOT[HULYDE Pp SaT[Ie} Sop UoTING 
-LIISIP Op SOOUDIEIJIP So] JUWUZTeS9 JUSNSNIII SoINSIy Sad 
‘£6 e samauedns uoynueysg J ap sino] 
-BA Sd] S9IN0} JUNT]UOD INb ‘JaIsed JatWap np «ayuTOd» ajuUR} 
-Joduit | elonbreural uC ‘sgULIOJsue]] UOU J[eUIZLO dT[9Y99 | 
SUPP $006 OP INaSIE] QUN JUTTVINe p INosIe] Op SogULIOJsUR] 
SOQUUOP ap SJaISed So] ‘ISUTY “$O00 Z 2P 3° $000 I 9P 189 
Opnid | & So[BUIZLIO sagUUOp ap SJOISed Sap oT[Te} eI onb eIp 
-USIANOS 2S UC “(OS%($O00 I/$000 SZZ) ETBUISIIO sTT9Y99. | 
sup $OSTZ [| ANeBIR] Op SJOIsed sap eB JUspUOdsaIOD (¢ Ines 
-IR] OP SPULIOJSULT) SIOISED Sa] ‘a[dWIaxXa 199 SUR] ‘(¢ OP JOIS 
-89 Op INVZIL] OUN JdAv JUDUIOIIVOUI] OULOJsue]) UONQL) 
-SIp B] Op oWMUILISOISIY | oJUSsoid Z JINBTy BT “$O0O STZ OP 
*G un oak “$OET LOT OP 189 UONNGLNSIP 9190 op ouRIPSUI BT 
[‘sesuo]e UoNNQL)sIp oun .p 18k _,s [I,nbsind ‘uonejuesgid 
RP] Joypioey Inod juouraind sistoyo 939 & JaIsed Np IIe} P| 
-RION] $000 SZ OP JOISed Op INIZIR] oUN Sede ‘[OUUOTJUSAUOD 
JUSUIZOULUTJ UN JURUUDAOU SONPUDA SUOSTPU Op s[eUIZLIO 
UOTINGUISIp ek] op oMTUIeIZOsIY UN dJUasgId | DINSIJ eT 
“SaguUOp op sina]eAa 
soyjed op jnopns sanjtsuOd UOTINGLISIp ouN JUdTSNIII p 19 
€ SOINSIJ Sx] ‘sopuess sat suoneAlasgo senbyjenb suduwi05 
A ‘saguuop op SingjeA op [IeJUdA9 VBI] UN JURLOdWIOD UOT 
-NQLjsIp oun jUuaNsn]ji 7 39 | SINS say] ‘suONNqMIsIp xnop 
Inod jusuIs9R[ dud Ing] 39 SIDISVD Sop INIBIR] LT Ins sITegU 
-I] UOHeULOJsUe e] Op JOJJo,] JUOMUOU p B | SOINSIy sa] 
“UOHNQUISIP RP] Op SULIOJ LT 39 S[[9YS9, | 
Josueyo op jUaWaTIOeJ JoULIEd steul ‘a][aYyo9,.p sajyuRIsSUOD 
" sap 3so nb 99 nod nsooe ajqeiduioo {ree un asoddns yusw 
-dJTegUI] SOQULIOJsURI] SoQUUOP Sep UOTesI[NN, J ‘soguuop 
sop puodgp Jaiseo np Inasie] e] :sgWULIOJsue] UOU ZI[2Y499 | 
inod piepueis sed juos ou ‘G6 v somnaLtigyul soguiojsuen 
saguuop sa] Inod sagsiyyn «prepur)s» JoIseo op siInasIe] 
SY] “UOT[HULYI9 | op JUepuddap JoIseo op IneZse] oun Inod 
Q19}UL,| Jed dANOUW 9}9 & dIIVOUT] UONPULIOJsUe) eT 
"PAS 2 
SUBP 9]U9A Op xd 9] nod JoIsed np ayeurdo Inasie] ey op 
JUSUI[Qeqold sJaJJIP ISOMPIJ] 2] SUBP B]U9A ap xud 9] Inod 
Jose np seumndo ines] ey ‘oyduioxe sed :\uoulasse[o ap 
S[QeLILA 9UN Pp SINIIA SasIOAIp Inod JorayjIp ined Jatseo np 
gfeundo Iinasse] e] ‘snid aq ‘a][ayoe ,p JUSUIASULYO 99 Ja}9]J 
-OJ JRIAOP UOT[HULYIY,| SUP SIOISeO Sap JUSWUI.e|dWI9/INE3 
-Je] P| 39 ‘sdu9} Np [ne snjoag uONNQuUIsIp e] “UaANOS ‘aN 
-ne | @ UOT[HURYSE UN,p 9jUR}SUOD JUSUTAITeSsao9U Sed 9}S91 
aU JoIsed np sjeudo InaZIe] eB] ‘“UoWASNoINoYTeJ ‘Vou 
-BA P[ OP UONRUWINSS,| Op VIT[IQuIS B][ 19 STeIq a] AUS 21qI] 
-Inbg un q1]1qQe}9 JoIseo np «apeutdo» ay]1e) ey ‘uoneodsayut | 
B J[QUNQLIe UONUITSS | Op SIeIq J] JOJUSWIBNe je} SIOISeO 
Sop INIBIE] L[ Op UONLIUOWSNe J “SopqeySUI JUOS DOUBLIRA PI 


SLI 


ap suoTJeUIT}sa saz snyd yo ‘(] InaBIe]) aINAd 4so sJoIsed sop 
Inasse] eT sntq ‘setidoidde ax1j Jotsed ap InosJe] e] JOUTW 
-JO}EP op ‘STOJaINO} ‘aIIOIJJIp 3S9 [J ‘UOONpold vy] nod saygid 
LA Sep 39 suoneunss sop Joredgid ap youlied souvipaul soy 
Jo10QeIE Pp UlJe SO[eUIZIIO S9QUUOP Soy JOSI[NN,P Wej VT 
‘aed yeses uoT[HUeYIE || 
op oURIpsu ev] Jo s9[OdJa}UI oURIPsUT oUN oTUS SIITIG 
-BLIVA OP DOUDIOJJIP LP] ‘Sed 9} UN SUR “UOTINGLSIp ve] op 
INdLIgJul Joiseo np syooiddes say ang welnod oueipeul P| 
yuaUod Inb JaIseo 9] ‘(apHenb sstuaad 97) 'G ans aseguly 
UONLULIOJSURT I WEPUOJ UO,| IS “OURIPSW ve] Op guBIO[S osnf 
‘6 onb sopuess snjd quos sajo,nb sored oueipeul ey Jour 
-1]S9 @ sed JUDAIAS OU IOISeD JOIUIAp 9] SUP JUSANO) Vs IND 
saguuop se] ‘(sonbruou0s9 saguuop sap sed 9] JUSTUAT[ONIIG 
-eY 1899 sUTUIOD) sonbiyauIAse JUsUTIaATHISOd no sanbuyauAs 
SUOIINGIISIP 9p UOTJOUOJ Ud ONS5UOD 3s9 sINpgd0Id 3110-0) 
‘sorjne soy onb pueiz snjd dnooneagq jsa ‘uondaou09 
es red op ‘Inb Jorseo un suep goe]d juR}9 SogUUOp Sop 9}Se1 
3] ‘aje89 Indie] ap ssatseo X ua *G &Q op [eUIZO UOTIN 
-ULY99,| OP UOISIAIP a] dus e] e puodsal0s ainpegdoid syed 
[Px amofe uo 19 (000 I/ Cc X -'X)*G) sed agiodiaqur 
dUPIPSU P| aI[dy[Nu Uo ‘ayeUIsLIO JT[IY99,| UOTAS SATVUINSO 
gueIpeul oun s1uaj}go inog ‘(?y—‘y) op uonnamsip ey 
ap sued e ginoyeo aso (“Px -'y)'G 39 un aupso.p onbnsneys 
esa vy no “(x-'y)'Gyi000 1* (Px-'X) = LX sIoTe 
aSI[N Uo ‘(JOU NUSASDI Np UONNLYsIp oun ‘o;dwiaxa sed) 
SOAESOU SINIRA Sop sWOdWIOD UOTNQUISIp eI IS] ‘afeUIsLIO 
J[Oy9,] UOTAS SANUISS sULIPsUT OUN IOUUOP B UOdeJ 
2p (000 1/*O) zed sardnynwi isa , x np ag[odiojur ouetppu 
eT (17 uoNoes Be] eB SILO9p UOT[HURYD9,p sprod np 
}9 S9QUUOPIO SUONRAIASQO Sap SpIe | B SWII}S9) UOT[MULYDE | 
ap uonNqmsIp ve] op aftuenb swaision af 1s9 “GH no 


7O. x! et! 
Rr ouies apes 
:SNOSS9P-19 JITVSUT] UOTJEUI 

-Jojsuvy vy e [odde yey suOAe SNON ‘anbysugjoesed onbeys 
Inod gsi[jn sioye JULIa SIOISeD Op plepuL}s s_quiasua uN 
‘plepUr}S B]]OY99 UN UOTAS S9QUUOP Sap dIIVOUT] UOTPULIOJ 
-sued] oun soOdwi0d uondo sulaIxnep eT ‘sayeUIsIIO sagu 
-UOP Sg] UOTas opnyd | B GaT[NOWed onbysiugoeRIed e] Op UOT) 
-OUOJ UD SIDISVD Sap JoIOgeIE B 9}SISUOD UOTIdo aIaTWAId eT 
‘uoneodiaqut | Inod (Ja1sed ap sInaZIe]) s1IOZ9Ie9 ap JIIIeI 
RP] op a8v[sal 9] Inod suondo xnep sJgpIsuod SUOAR SNON 

XTdA [2etorsoid 9] nod sesipyn aiesuty UoNeods9}uUT ,p 
IPOYIOU! LI JUSWIITeS9 S99 Jd ‘SaTTONSUSW SUOTJBUUNSS $3] 
Inod JOS kB] ep uoNonpold ve] Inod soueueA ev] ap uOeWIN 
-S9p Janjoe sWa}sA$s 9] INOd dasI[NN spoyIoU PY RI IS9,.D 


) 


“QURIPSUL BP] JURUDUOD JoIsSed Np INasIe[ ke] = 2 
(QUIPSUL BP] JULUDJUOD S][PAIOJUL J 
ap uoneindod ve] op sjusWIg[9 Sop JNLUITSS [210} 
SIQUIOU) DURIPSUI aLIOZ9Ie9 ek] op aouanbay ey =f 
OURIPSUI PL] JULUDJUOD JoIsed a] JURpgooid 
SOT[EAIOIUL So] sno} Inod saNe[NuINd souanbelj ey = Jo 
uonerndod 
PF] OP SJUSWIZ[9 Sop JBUINSE [eI0) eIqUIOU 2] = VV 


000g aiquiacep ‘a}egnbua,p senbiuyos 


oUPIpSUl 
RB] JURUAJUOD JoIs¥O Np sINSLJUI aw] eT = 77 


uoT[HULYD9 Pp sptod sap Ins s9xe 
anbnstigjoesed el op saNe[NuIND souenbey ey = 4 


(1) eee +1 =(N)f =I 
DON. 
be 4 

‘UO][HURYS9 UN .p oURIPsUI PI op alleou 

-I[ UorNefods9jUI | Inod snossap-1d s[NUIOJ eT WUINOJ 
(ZS61) JJMIPOO AM “UOT[HULYI,| op sULIPEUT eT JofOdIa} 
-ul A Jo UOT[MURYD9 ,_p SoguUOPp Soy Jodnol3 Be a}sIsuOD 
uone[ndod oun .p sueipeul P| JoUINSa.p UOSv anne ou, 


(1’€) 


oregul] uoNejodiayuy = *Z 


1210} 
sptod np gntoul ev] assedap spiod sap sgroosse auIWOS 
Ry oyjanbe] inod uoneasasqo aigruaid ey] Jauuonsefes — 


‘sgloosse aygnbua .p sptod sap swiwos e] Joynuinsoe — 


‘Juepusose 
dIPIO |. SUBP UO][NHULYSE | 9p SUONVAIOSGO So] JOL] — 


:snossap-1o sedejo 
$3] Ins Bpuoj as ainpgdoid anaD (9661) OFYS J0 OLY 
SUBP 19 (2861) NA 19 OLY “ILAOY sup s9pUeUTUIODII 
4SO dIS9eNS 99D “oURIPsU eT JONIIS ap JueyoULIEd 
uoT[HURYd9 | ap sptod a] ‘sagdnois uou saguUOp ap 
Jned & Uuol[HULYd9 | op oULIPSU PT JoyNdTed & aISISUOD 
uoyeindod oun.p oueipeul ke] JowNse,p uodey ou 


uoT[HURYS9 | 9p ouRIPyN 


JULIPIUI LB] IP UOILUISI,p SaINpII01g “VY 


AISOTOGOHLAN *€ 


‘dN 39]OA 9] Inod gsteiq uou sprod 
UN.P 32 ANS 39]0A 92] Inod wWoLa\sod & syneNs sprod un,p 
apie | & Seg[nojeo yuOs saTfa 39 ‘sadnois GN 10 qN§ suoy[y 
-ULYD9 SOP SoONUd}qO JUOS JULIPSW ap SUOTIVUINSS SIT 
“apNIIIIO DOAe AN UOT[HURYY | SuBp 
SOSNIOUI JUOS SOANDU JUDUISZO] Op s9ITUN sa] SoINO], “stuULIEd 
SUBS SJI}IDV SJUSUISAS 9ZUO0-}9-9]URXIOS Op [IO} UN JUSWIZ] 
-[anjoe 21STX9 [J ‘Gq Spuvis so] Inod xnap Ins uoT[HUeYo9g 
-SnOs UN 911} B UO NO 4UdUIZIS-sNOs UN IStoYyd & UO 19 Gd 
SPpULIZ SUILIIOD YSIAIPNS & UO ‘sINOoIed Z| SIINp9l op UIFY 
‘(sjuouises sgjoadde) sguuoy[nueyss Gq Sep SUIWSY9 sap jo 
S$9INOI Sap s[quIasua | JUdINOOIed xnedO] s}JURJUSSgIdaI Sap 
‘QqN uo][nueysg | ap a8e1 np sidep OWIgISION 9] INOg “0861 
op JUaWISsUd.ey Np uoTendod x] op sI[Ie} eT UOTES SIFHe.NS 
‘(dd = 086 ep JUeWasus.0y Np JWOUeIQuIOUEP ep SIL} 
-SIp) suorZe1 sayned op synens onbyeusisXs uoT[nueys9 
un 1s qN UO][HULYDE,| op adeIN Np gIsap sUIgIXNap oT 


“sed quos 
9] OU SgIZap SUIgIXNep jo JonUeid sap xnv0 ‘sIOU a] UOTES 
Sjuepuadgpul JUOs gIZap SWIISION Np suoT[NURYDE sa] “opm 
-11199 DOA® SNjOUr UOS snqd no saytuNn burs sp UoNONNsuOSD 
9p stuued sap ‘ajuerenb nod un adA} ap [egos uoT[NURYDE 
uN ITU9}qg0 & UOSeJ op soytuNn oNenb ev sun.p JueyOduI0D 
uONONTsUOS ap stuLied sap juowonbyeugisks sUUONDI] 
-98 UGC ‘stuled sop youl9 Inb guuoy[NUeYse WoIpus onbeys 
ep stulied ap xnvong sop jueuaaoid UoNONNsUOD ap stuLIEd 
ap onbrneuisiss uoy[nueyse un yuoUUONDeT—as (TY) xneoo] 
sjuvjuasgidal sop ‘stow onbeyo :juewlayjansusw gyndexe 
389 ANS UoT[Hueyog | op sse1n np siZap swWiaIston sq ‘gIZap 
suIgIxnap 9{ Inod 9]1uN [Nes ouN 91 B UO ‘sed xneIquIOU 
ap sueq (78 ‘18 ‘8Z = 1) fT eguue, imod stup-sieig 
xne sig sted sop [e}0) a] 19 T spuue,y Inod stuned 
ap UOISSIWIA | oUe Woddel np sgigpuod suueXou! sun ,p 
UONOUOJ Ud SaTJHENS 939 JUO SOIPUS sa_ ‘(etuUad~p Jed sto} 
oUN 981) UOT[MULYDE,| Op AdqN Sep IneweyuI] e sted 
Sap JULOUIN s}OIpus.p syHens onbyeursysAs uojpHueyse 
UN 4S9 qNS UOT[HuLYSg | op odeIN Np gIZep oueIxnap a] 
“Jd oun .p snjd yueyoduios sayens 
op imsed & (Ldd) [te eB] @ offeuuonsodoid gy1qeqoid 
oUN JURUUSAOUI S99) 919 U0 (YWN) soanejussaidaiojne 
uoU AFdN Seq ‘(AdN 3uN,p senjysuod jsa ayeNs ey) opny 
-IJ99 DOAR UOT[HULYDE | SUBP SaSNIIUI 939 UO (WY) Sea) 
-ussgidoiojne Yq soq ‘ayens red Fqr oun sin ve UG “uIe} 
-TTOdoNEUI 319}deIVD DUIQUI 9] JURA IeIq OWQW NP AdN.P 
S2QNINSUOD JOS sajeNs soy ‘a]qissod np ainsou ey sueq 
‘sTuliod sues soUuoZ sop suRp s]UDUIEZO] ap asejUsoINOd 
3] 12 ZR6I Op Sosapuod sfaruoprses sted soz ‘snd 39 sue 
OI SAP OX6I Op desepuod uonejndod ex] uojas uoIsaI eT op 
INILIDIUL,] B SI9TJIENS 939 UO Yd sv] ‘syuepusdgeput juos 
SUOT[HULYIE Sd] 9)SOI J] Inod steul ‘qqp SoWiguI soy JUsyIOd 
-WI09 saygnbue xnep soy ‘ISUIW “SOIUIJop UdIq Soy] sop 
jueke sajusoelpe uleLia} ap satjoored sap yuos nb ‘(Aaains 
uonendog jUaLIND) SdD P| oP O86I ep uerd np (qqn) eseu 
-UOT[HURYSE ,p SorrewiLd soyluN sap UOTTMUYD9-SNoOs UN jsa 
dN 19 dNS§ suoj[Hueyse sap o8ein np sisep Jorwueid aT] 
"SOJUDA Sd] 19 SSADYOR XNeAavdJ} Soy ‘IoUeYO 
Ud SASTU Say :aygnbud.| ap sgjo sonbysnyejs sol SOAR 99A9[9 
anagid UOe[a1IOD ouN yuRAR SO[qULIBA Sop UOTES SoIJNeNs 
sgiZap sinaisnyd e sayst{iqeqoid suo][URYSS Sop 1UOS xnop 
SX] ‘DOS 2] ep uoneumnse | op oned opueis snjd ej omNsuod 
dS 8] op uoypnueysg TJ ‘(qN) Aeaings uueduoy sygnbua | 
‘uonefndod-snos swigixnap ek] 2p Se9Jl} Set[99 19 ‘(dqNS) 
SIULAg JO asQ) Oy} Jo Adang sg[NINUI sJgnbus,| JoUUsId 
-wi09 uonjendod-snos aigruaid e] ap $9911 DOS P| op oseu 
-Uo][NULYdE ,p seytun say ‘sed Juasrx9 usu INb safes] sau0Z 
sap 19 UOTONsUOD ap stuLied sop juasIxe Inb safeoo] souoz 
sop :suone[ndod-snos xnap JUSIU0S DOS B| ep SIOATUN /T 


QOS VT Ad ADVGNOS AG NVTd “7% 


‘9 UOT}9OS ET B JUSANOI} os SUOTIeP 
-URUIUI099I SOU 19 SUOISN[DUOD SON *¢ UONDeS P| B ag]UESgId 


sueipaw e]UeA ap xd sep senbijded sed soueUeA &| BP UONEWUISS,| }9 UONEWIISS,] suUBWUBIS Ja UOSdwoY rll 


€EZOT ‘OC UOIBurIYyse AA ‘snsuaD oy} Jo NeaINg ‘S’f) ‘UOISIAIG SUIWIUIeIZOIg pue spoyJa|A] [LONSNLIS DTWIOUOD ‘ULUISIS “Ss preyory Jo UosdwOY], ‘f suLAWEY 


UOLINUIS op epnia,| PANOW yUO INb DOS ke] ep soguUOp 
op stow aenb sap sanbuidurs s}ey[ns9i Soy WEP p UOTIDS 
Be] “€ UOTOes eT e sanbi[dxa 3s9 s[e1oUgs sISO;OpoujSW 
eT OOS 2] op esepuos ap uryd np ndsede joiq un Z UoNses 
PR] & PIDANOI UG “alleguI]T UONRTOdI9\UI oUN INS soaxe SUOT} 
-eLIvA buld 39 UOTTHURYDE,| Op oURIPSU eT ‘SHI uoNNedas 
OP 2Ppoyieul | op nud} a}dWI09 ‘aueIpsUI Le] Op UOTRUINSO,p 
sopoyjeu xs suoreduio0s snou ‘gsodxo jueseid a] sueq 
‘asueyo suoT[NURYdE 
sop uoNNqsip ey aonb ainsou e sdwia} a] suep Jon[OAg.p 
juonbsi 39 SUIeWIOP 9d] UOTIS JOIOJJIP UOANd sIoIsed sap 
xneuido juswsor[duia | 19 InsssR] eB] ‘slojaynoy, ‘sonbryew 
-IOJUI SadINOSsal op sulow! dnooneaq a3I1xa 39 UONeWIIXOId 
-de quuog oun yuINOJ UONR[OdIaUI | ‘oUeIPsWI eT] JURLIOd 
-WI09 aT[eAIOjUT,| Inod suuojiun said nad e yWos saguUOp 
Sop UOTNQUIsIp eB] aonb nAInog ‘sueIpsUL ev] JURIOdUIOD aT[eA 
-19}UI J Inod areauT] UOeTOdIjUT OUN JasI[NN & Jd (SIOISVD 
sgjedde) sjalosip soy[ealajuI Ud sanuTUOD saguUOp Sse] 
Jodnols & d]SISUOD UOTLUINSd Pp PpoUISWI ayNe aU_) ‘“sQUUOP 
JUSUIasse[O Op s[qeueA ouN.p Ino[ea onbeyo snod sjoun 
-SIP SI} Sop a8Ixo [I,nb sored asuajqul [Noyeo ap [reaey) uN 
JOSIX9 Jnad UOT[HULYIE Pp SOULIPSU Sap JO9IIP [N[vd JT 
“snsuad ou} 
jo neaing ne g1oqeia uoNNEdal S9AR DOURLIRA P| Op UONRUW 
-1]S9 .p Ja1orsoid a] ysa inb ‘(GEG Avy ‘AdAInS Ka Tqulog 
WOJJ SIOURLIVA) KX Td A UP AND Ud SIU JS9 BpOYIOU aIQIU 
-Iop 99D ‘ouRIpsU Be] op UONeWITXOIdde oun I1Ua}qQo Inod 
sosednois sonunuos saguuop op oileguly] uoneodiajur oun 
B [odde j1ej spoyjowl oWIeTXNep eT “aANeTNUIND UONNGLYsIp 


vsn 
I 


ap sonburduis suonouoy op opie | & oUeIPSUL eT JOWT)so Inod 
uo][HUeYE .p spiod xnv jadde yey ssgtWoId ey] ‘oueIpew 
PR] OP UONLWINSd,p SOPOUIOU XNOp SI9PISUOD SUOAR SNON 
“SHIN 2dAy 
ap uonnedal ve] ap nus) a}dwi0d ‘aoueLIeA P| Op UOTPWISd,.p 
yo uOneUINSS,p sgiatidoid souuog op juRAe sURIPSW ve] 
OP UOTJLUIN}SA PP SPOYJSU SUN JOANO] ap jessise,s |] “sonuy 
-UOd saguUOp Sep Inod ouRIpeu BI op UONeUINSd,p oIsO0[Op 
-OYJOUW P| JOUTWWLXI9I BP SPlO9P SUOAR SNOU ‘AN[OAd DOS PI 
Inod doueLIvA ek] Op UONeUINSd ,p aIs0[opoyyow ev] onbsing 
‘(8661 uosduoy) uoNedel s9Ae sOURLIRA ke] Op uo 
-PUINS2 Pp SHI epouieu ev] Jed agov{dual elas DOS ke] Inod 
OURLIVA P| Op o9}9d91 UOU UONeUIT}SS Pp a][aN}Oe aINpgo01d 
RI ‘R[-JUSWIOUI 399 Y “(SqAIS) WaIsAS BUISSdD0Ig DTUIOUODT 
poziprepuris 2] ‘grueuior ouleshs uos eB QOURLIeA BY] 
OP UOTLUINSS Pp 19 UOTJBUITISA ,_P sjanjoe soula}sAs SOP DOS 
RI Jassed JQVUDIq IOI SNSUdD 9Y} Jo neaIng IY] ‘q’¢ uON 
-9as BP] @ sagnbijdxa juos Inb suostei sap inod (uoy[HUeYE 
“TWep deAR sgI}Ipowl uoNNedsl) SHI epoyljsur e| 
B UdIg aJgId as BseUUOT[HULYDE _p UL|d J] JUOP sgIZEp sINeIs 
-njd e aystjiqeqoid ajgnbusa oun 3sd QOS eT ‘((uonoNysuoD 
Jo A2AINS) DOS saenbus,] ap siped a] suep suosieul ap 
SINIJONIJSUOD SOP MIIAIOJUL.P IIOA Jed sasinboe saguuop ap 
Inyed & sag[Noyed JUOS SUOTJBUIT}SA SID “sONPUSA SUOSTeU Sop 
Q]U9A Op xd J] JURI9 VID UOTJEWINSS oUN ‘SoANdU SUOSIeUL 
sop sonbysugjoeses sinoisn{d Inod suvipeul e] op suotyeul 
-1)s0 sap atjqnd stup)-sieyq sap snsueD ey} Jo neaing a] 


NOILONGOUINI T 


‘UOHONSUOD Jo AIAING ‘UOT[MUY9-TWIap deae egyIpoul UONNHEdgl ‘ouIPIW :SA TO SLOW 


“QOURTIJUOD OP SIT[EAIOIUT SINI[[IOU! Say 19 dITTIQeIS SINT [OW 
RI ‘QAP SUIOW 9] STeIq a] JURLOdUIOD VoULUPA PL] Op SUONeUIISO sop JUQUUOP saguUOPp sap pusdgp aI]Ie} e] JUOP SaTTeAIOIUI 
Sd] ONb YAIOSGO SUOAR SNON ‘SOX9UUOD DOULIJUOD OP SA|[LAIOJUL SOP DINLIDANOD LJ Jo QOURLIKA P] OP SUOTIBUINSA Sep sITTIqGuIS 
PR] 19 Heo sterq a] onb oulgUI op dUeIPSUI BE] 9p SUOTeUINSS Sap soUUDAOU SaNTOSeR sINaa sa] 19 souUdXOU sonbNyespenb 
SINDLII SI] JUTLUXD SUOAR SNON “QOULLILA P| IP SOXOUUOD SHI] SUONLUITISA So] INS Ja SAULIPEUI Sa] INs saguUOP sap puedap 
O[[fe) VP] JUOP SaTTeAIOIUI S9AK JUSUIDdNOIB J] 19 OXY QI IIe) Vp SaT[eAoyUI DOAv JUaWIOdNOIsS 9] ‘(UOT[HURYDE ,[ op DULIPZUI P| 
duOp) JUSWIAdNOIS ap soUaSQe | SP 32}J9,] gIedui0d suoAe snou ‘sagnuUIs satu suoNe[Ndod ap yo aygnbua ,p saguuop ap opre | 
Y ‘oueipoul ev] JuUeOdu0d saadno1s saguuOp Sap d][eAIOIUI | SUBP SESI[NN JULI9 SITeQUTT UONe[OdJa,UT oUN ‘souRIPsU sop 
UONeUITSS,| Op aNA Ud sagdnoJB saguuOp sop JosI{NN ke ‘s_NoTe9 sap ana ap utod np ‘saeueAe snotsnid uepusdso arsrxo I] 
‘QULIPSU SUN ,p IdeUUOT[HUeYS Pp POULLILA EL] OP UONLUINSS | Op SNA Ud Sa}9daI SUOT[HULYIE Pp dULIPSU! BE] & SHIN Spoujew P| 
op uoneordde | yuatndde sgtjqnd syuouinoop soq ‘doURLIRA P| Op UONLUINSS Pp (UOT[HUeYI9-MWIAp ddAv JeIJIPOW UOTHed91) 
SHI epouleul eT & usrlq aj91d os aseUUOT[NUeYDE _p ULTd a] JUOP sgIZap sinotsny{d ke ajsI[Iqeqoid aygnbus sun 3sa DOS 
eT ‘(uoNINsUOD Jo ASAINS) DOS eenbus | op sped J] SueP SUOSTEUL OP SIND}JONIISUOD SOP MATAIOIUI ,P 910A Jed sasinboe 
soguuop op sijred & sag[noyeo JUOS SUONRUIT}SA SOD “SONPUDA SUOSTeUI Sap 9]U9A ap XUd 9] JUR}9 VID UOTJeUIN)Se OUN ‘soAnoU 
suosreul sap sonbustigjoeses sinatsnid mnod ouetpgui e| ap suoneunsa sap at[qnd stup)-sieyq sap snsuaD ay) Jo neaing a] 


aWnsay 
INVINDIS *S CUVHOIe 3? NOSHNOHL ‘ff ANINAHLVY 


SONPUDA SUOSIVU SIP SULIPOUT 9}UA Vp XIAId sap 
sonbiydaa 1¢d sdUvLivA B] 9p UOIVUIT}SI | 39 UOIBUISS "JT 


eZLl 


epeuey aenbyasneis 
E8L-ELL “dd ‘ZU ‘9g IOA 
000z esqwieoep ‘ajanbua,p senbiuyoo | 


one ee 

ashi Bue ante 

oa pase te 

‘ olin 1 WE Rete wen yy | 
hy Oe Pa 40 a" ous Tf! 

ci Puy, aay. arpa’ Dis ch ot bjaese 

ap iad whigey nia oe co Be, hy ' 

Mage , 


we, 
eve Le ee 
ia a) 

Weer TR 


ae 


i. i) 


v4! vy 
sonae 
! a dle 
re ¥ yee 


Peete / ’ 
ys eine a “et WIR WD, "ay 
i bay 
wr mi 
* 7h 
Fl «as. : 
Neb bie iw 
hs ac fa : 
eas 5 
panna Pear cat oy hie egg se 
peach spins Eki abe tty . 
Besecad i ty Sm a OL 
i yaa iN 
‘ Need! we , 
yeoman eae 5" ets 3 
ferry 14. sleemetanet 4! cau ign , 
FTE MERE OA a VERIO! per im 1 
Bee wae PNG RPVAL Pa re 
aloe Mii a Oag Ain Ques youl rer 
Wee ONG hok a OST Ke 
Me CAR SM Aa! KO PLE 
Pear iM ora WMI, as MG ah etinn pM 
- BAG ' idee Mt iy sh Or le a 
ont Hipster ee! Pe yer 


a PATE: ARE TE Oe ot 


fu Weniat se i Lx: ° 
a ‘ahed . 
nei Ng te fae A aes 
a aerapingiebon, 
; fal al, i ft ied 


reestehvoiar teeta exc 


: we on re é beet.) + 
s, 4 
Py 


ay 


ae a ee b Ee 7 ott cece S Ls 


ity v - ’ 7 


abraham, — sanyurionssd uss digg hea 
fs ere ee anes yp <a ay ne 


BY 
aWoromar's RAM nvyl Ota 
4 wr uO ony SPOS Ey 

i) * al, 1 


‘en iw ea iv ee He sp Py? 
“vai ‘ ai gipe 


* 
J hy 


Fite SWRI dete FY Sod Soe ieay igade 
+a tre 
ie li pa Sra Allee 
pe ere are bal ree y 
oP le: Watt 
“ae hoy Sigh ety ashearavar aclghy 


ame ty Pr: 


a Ary 96 Sy ay} 


7 we i loner gue aN Ty Pp ey a wie riws 
aPoA Ph Met Cia PRET ues ree 0 a creer hg 
PRGA: Gy Tat Oelh, SAR ES MR ey WT Lig FIT 


bh ithe nape wee Asveeererry yoy? et 
og > OA Sy: s3 ya) AVRGT ae i) “ 
Sigg te 9 Er lye 


NP Pee FONT ene 
5! “thts! 7 Tre \ 
7 isan ite Wn, ys O 5 


a 


“ 
meh 


21 ae 


4 WANA? eta preety ht? ‘ aS) yes 
Trwitte? af’ } 1 *% ry ae wit, wh +5 ) Mb) i 
> Bama f Vy aPA ¥, } Le “ry i. 3" 
( Pe yay i rt 


\ 


EMORY i riderilt PPR 
Pet in) Se SALE = me wid Use e! I abl pry ye y 
? 7c i , y 
wr > 4 ‘ . - 
7 ray S i ee! t mahi ie | file oe 
a) La z 7, 3 LnGh ¢ 24 te cil “ ; J ‘ a 
h j 
¥ iQ yi a fn aad tert ore | Bint avian NG ve pi 
Se BEAM iat OP oy ar TS oan : 
? aT . a . 
ie > i 
, wy. i fire ie) - 
q 7 A 
ee a ‘ a ! 7 4 Ay Eton ee eR 
f Fe arcs | 
het ee i eg re . | Midlt Ce Piss res 
\ )% 
: 
Ys. TF ke . 
r 
ie] L a - was tA as ere 
il it i rl aa 4 iL Lite € lke abe 
F aig ts % wr i , Te Sse Ne EK 
¥ ¥ 
4 at ipa ¥ Fis J P 
ed : , - . 
bu mse herr 2 a 4 wT Pel: f 
} h ; 4 * ’ oe 
i od ba, beget td § itd na EY yen oe 
2 i 
yd 1:Gu ‘4 yepy Ve pe PR 4 
p 
wis 4b CFD WIRED. 2908 AC a} ee , ae ; 
‘ rr err? Pree vel ie th Ly arts ef ee ey 
. = - “4 a> Crit me A gates A Son 
EP CEr Th eh ee a 
oy ' ; ' Pj 
* CU CMTE U- fw Wn p14 wy 
¥ i) Sg py Pusremr er gel Sue 0p : 7 ih Pg 
LA MRS MER ER, COREE A iat ey Oop Petes 
wh Le Ar , jogo yale , + iy 
4 : wai . = Ie +h Ta (ew bs ae f) OP OS Ty 
rorya fea cgue: Ta iruel hy rah 42! raat. 
’ 2 
AAD piety eg (af afe a a Meee iia 
4 Sa 
; . ¢ hoo Ay) tee Sea 
oie NESS FET Me Ok ee 5S, 
f 4 i Pe « 
vf) EY TOA ROR Set of!) PRY mh 
* aa ) y as, . 
a 4 , ‘ 
rb vd ' C. . . f a) < 
f 
Cn SH yi M4 argue p' dented: 
ea ue he ota mae beeepeatia 
- i . % 
Dame) eat 1S Aen UF OR ale ft TS Ca 


; | ry 


‘J'd'S Jolaas|q ‘wepsoysury ‘suissas01g 4o1apyag pup apnjiyiy 4of 
sjapoy Gyiqoqord juawT ‘stsdpouy jauvg (€L61) W'T ‘SNIDOIM 


“AUSIOATUL] BINGILL “eyed [esOsa}eD 
Jo siskfeuy oy} Joy wiesso1g [eIOUSH Y ‘WA (L661) ‘f “LNOINYAA 


"SUOS 29 ABTIMA UYOL >YIOX MAN “(SP_ ‘72 72 ‘B1OQATJ “T) 
‘A1JONO SsSa20d1g pun Juaumainsvay Aadaing “eyep O¥IeU 1OQe] 
0} uoneordde ue ym sAaains Joued ul JoUa yUOWOINSeOW pue 


asueyo sunesedss (1661) ‘YU “ANIDHADNVT 9 “A “TOd Ad NVA 


“IpI-SIT ‘7-1 ‘ST “yoauvasay 
pup spoyja 102180]0190G “JOU JUOUIOINSLIUI IO} 3991I09 04 
[opowl AoyeW jusIe] VY “(9861) ‘f “MNAAT Ad 39 ‘Ad “TOd dd NVA 


“290JO 
SUNULIG JUOWIUIOAOD :'D'q ‘uO sUIYseEAA ‘¢9 Jodeg eotUYyoa] 
snsudd oy) Jo neaing ‘§'qQ “Asojopoyjaj, pue usIsoq :Aoamns 
uonefndog wend § (0007) SASNAO AHL dO Nvaund ‘s’n 


‘20NJO SUNUL JUSUWIUIZAOH 
. “SN YO'd ‘uouryseM “¢-YL-dSI-GLS “Butsnoy pue uonejndog 
Jo sasnsusp suneneaq (C861) SASNAO SHL AO NvaUnd ‘sn 


“88P-8LP ‘OEP ‘06 “UONDIDOSSH JDDNISUDIS 

upridaup ayy fo joudnor *KOAINS 9dJOJ INOQe] uUeIPeULD 9) 

WOY ejep 0} UONLoTTdde YIM JOM UONLOIISse{O JO} SayeUIT]SO MOT 
sso1s Jo juounsnfpe oy1 UQ (S661) “A'N'f ‘OVA 29 “O'V ‘HONIS 

“EST-ILI °@ ‘p% ‘atgnbua,p 

sanbiuysay ‘sgyqnd a8ewioyd op xne} se] Ins soUaptIoul Ina] ap 

asAjeur 39 9AtjOe UONelndod ey] ins aJgnbus | suep UONLOIISse[O op 
sIndia sap suoneUNs| (8661) ‘f “HLYIA.LSV9 19 “W “MIWTONIS 

"696-196 ‘16 ‘UonDIo0ssy 

/DIUSYDIS uDdLuauUPp ay fo jDUANOL “eYep 9dIOJ JOQR] 0} UONeoT|dde 

‘spoyjoul ADAINS MOIAJOJUIOI JO SSOUSATIIAJJO oy) Suyenjead 
Joy sempesoid UD (9661) ‘f ‘HLUIM.LSVD 3° “W ‘XIV TONIS 

‘7679 ‘1 

‘LI ‘yoapasay pun spoyjap 10218001905 ‘yoeosdde ssejo juayey & 
‘sKoains [oued ul Joa asuodsal 10; BuNsnf[py (8861) ‘£ ‘AHAOOHS 


‘yodoy 


snsudZ oy) JO nesing ‘s'- feWIAIUT “(6L6[ JoueN? py, ysnosyy 
8L6I JouRNd puoses) juowedxg uoNeElIouocey yUepusdapuy 


SdO 24) Woy synsoy MoIAJNIUIOY «*(0861) “I “WANIDAHOS 


LZ 


'g Judy ‘7 uodoy Jeotuysay, weay, siskfeuy 
depi2aQ SdD ‘snsued ou} jo neaing ‘sq ‘Ailend eieq uo 
SIDIJJ_q ‘OMLUUONSEND Sd Ol} 0} SUOISIASY (7661) ‘f “‘AHOHLOU 

“erueAlAsuuad Jo AjIsJoATU:) 
‘JooyoS uOeYAA ‘JeIO}OOp ap aseyu, ‘suoneoljddy surjapoyy 
S104) YIM siskfeuy simonNg woe] (7861) ‘S'O ‘NASTNOd 
‘917-LOZ 
‘LL ‘SONSIIDIS puv saimouoaT fo malaay ay], ‘UOeOIJIsse]o Ul 
SIOUO YIM [EPOUL ISO] [EMOUT[NU & :suUOT}ISULI) JOYIEUI JOge] pue 
syyoueq yuouXopduiouy, (S661) “I ‘SUAIWIWNS 3° “f ‘VENFLOd 
"BEEI-GIEL “9 “pS ‘Sol4amouoag sorueudp joyreUl 
soge| pue siouie Sutodey (9861) “TI ‘SUAWINNS 3° “f ‘VaNALOd 
"PLS-ISG ‘SUOSg 29 A2TIAA UYOL :4JOX MON “(SPY “70 12 “JOWoIg 
*d) ‘SKAAANG Ul SAOLAT JUaUAANSDAPY ‘S\UeUTUID}oq pue UOTeWINs| 
‘sour A esuodsas ajduiig’ (1661) ‘0 ‘HOIVLYVAHOYINN.O 
TI-SSI “Z ‘p ‘sousuDIS jo121fO fo jousnor “Ayjenb 
asuodsas Asains pue smeys asuodsai Axold/JJag (8861) Or ‘SAYOOW 
‘O6E-S8E “E 
‘9 ‘sausIIDIS D1u0U0Iq pun ssauisng fo jouAnor ‘sotureUAp yoyxIeU 
-JOGe] PUB S[SPOUl JoOUa-UOHeYIsse[D (8861) “d ‘Ad ‘SUAAAN 
“Sp-LE ‘SousIDIg pup so1mouocrgy fo maiaay ay J, ‘UdWIOM 
pue usw 8uNOA Jo} sart{Iqeqoid yusuIAO[duI9-aI Jo s}UeUTULIO}Op 
‘SONYSIO oy) UT JoyTeUT JOgeT YINOK OY] (6861) ‘(WT ‘HONAT 
"V9T - OPT “TT ‘SOUSUIDIG [D4OJADYag pup 
jouoyponpy fo }DUANOL ‘S[2POUL SseID JUD}e] POISOU-UOU JO} BUD}LIO 
UOHLULIOJUL UONIITIS [PPO (L661) ‘W'D ‘NOLAVG 9 “H'L ‘NIT 
‘O-D Suryst[qng LNAN-SAd ‘uo\sog 
‘spoyjay ajpiapayjnyy 4ayiQ) pun sisdjoup uoissaisay paiddy 
(8861) ‘AM “MATINW 2 “TT “WaddNy “O'd ‘WNVENIATA 
“ILI-LOL ‘9¢ ‘sormaumorg *s1soy onsouseIp 
JO sayel Joe oy) SuNeWNsY (0861) ‘A'S ‘YALTVM 3° “T'S ‘INH 
‘O9E-9LE ‘bp ‘II ‘Youvasay uotuldgc oygng 
fo joudnor JoUOYOUAaIUT “BUIPOD JOTARYOG Wo Aj] IQeI]aI 3s9}01 
-1s0) BuNDIPeld (0007) ‘f ‘AUAHSNA 3° “A ‘YAONIS “f ‘SSHH 
“€87-LYT ‘Lp ‘vo1u0uoaq ‘aouapusdap 
aye]s puke snosussOsa}oY4 JO JEPOU sWIY snonuTUOD eB WO 
sJomsue pue ‘suonsonb ‘suontuyeg ZjuewAo[duroun oinjry asneo 


juowAojduroun so0q “(0861) f°D ‘SWfNO" 1° “Lf ‘NVINNOH 


000Z aiquiaoap ‘ajenbua,.p senbiuyoe |) 


“LL-S9 ‘sosneig Joge’T Jo neaing ‘¢"() pue snsuaD ay) Jo neaing 

"SN “O'd ‘UOIuIYysEM ‘S9uS1IDIG 2240.4 LOGDT Ul SMO] y SSOLD 

uo aduadafuog ays fo ssulpead0dg “10112 asuodsal Jo souaso1d 
dy) Ul UOHeUINse adueYD SSOID (S861) OL ‘VONHO? “AM ‘MATIN 

‘COE 

“6LZ “SUOS 2 A2TIM WYOS :Y10X MON ‘CSpg “Te 19 “Joworg “d'd) 

‘SKAAdNG Ul S4OLIY JUAUAINSDAPF “MOIAIOAO UL :MOIAIOUIOI JO 
sisdyeue pur usIsep ey], (1661) ‘I ‘MANIGUHOS ? “D ‘NVINSYOd 

TL-TO ‘TL ‘Ly “MaiAay Suonnjay 40qvT 

pub joLasnpuy ‘siosn yeodal Jo suoneInp jLyouEq Jy) WoL soUDpIAq 
G2anoIppe soueinsur juowAojdwoun s~ (€661) "W ‘NVUOO 

‘OP-LE ‘OIZ ‘uawasnsvay jor1B0j0yoAsg pup jouolvdnpy 
“SaTeOS [BUTLIOU IO} JUSWISOISe JO JUDIOJO09 YW ‘(Q96I) ‘£ ‘NHHOO 

“pI-8 ‘9 ‘Yyaupasay pup spoyjapy 102180j0190g “sisKyeue Jeaut{-30| 
Ul suratqoid uoUTWOD aug (C861) ‘Ss ‘NOSVITA #9 “D ‘DDO0TD 

“IS-9 “LOE ‘78 “UOUDIDOSSP [DOUSHIOIS 

upridaup ayy fo jouAnor ‘SMOTJ JOQgR] 0} pat{dde Jove asuodsal 
[erwoun[nuy Joy spow VW (L861) “VM “AATINA 39 “OWL ‘VAHO 

‘TZI-OL ‘Od ‘UOIuIYse AA ‘UINpUeIOWO;A| 

snsud_ 94} JO nesing ‘S°~) [eUIO}UY “sIsATeuY DTW JO} sojdwexg 
uonenuis § (6661) “A “YAOUAATAGNM }° “f ‘AYAHSN 

"SSolg OIWApeoY :yJOX 

MON ‘CSPq ‘uosIapuy “g'V 19 JUSUAA “Wy ‘ISSOY "H'd) ‘Yauvasay 
daaing fo yooqpuny] “\uoulenseaW “(€861) ‘MD “LGALSNYHOY 

“€CO-S16 

OZ ‘LQ ‘uoypII0SsPp JvO1NSIIDIS uDdI4aUp ay] fo jouAnor ‘KOAINS 

uoyeindog juan oy) 0} suoneotdde yWM ep MAIAIOUIOI 
Jo Ayjenb oy3 UO (Z661) ‘D ‘NWWSUOd 1° “d “YAWAIG 

‘podey yeoruysay, snsua_ oy} Jo neoing 

SA [eWIAIUT “SqD BU} 0} SfOpop AOWeP Jude] Jo uoneoryddy 
uy (L661) ‘d ‘NVOVNVTA 3° “f ‘AYAHSN “d “AAWAIA 

‘O€-E7 ‘OL ‘uONDIDOSSy 

JD2NSIIDIS uDdI4aUp ay) fo oUAnOr *SKOAINS [aUed WOOLY SayeUT}S9 
uo seiq dnoid uonejos Jo yajjo ouL “(SL61) “W “A ‘UV TIVE 

"€68-S88 “€ 

‘OL ‘Mataay a1u0Uu0s7”7 uvdidaup ay] ‘sousuedxa juowAojdwioun 
pur sjeds yuoutXojdusouy °(0861) ‘W'D ‘NIV? “WD ‘SOTYIAV 

"SUOg 7 ABTA 
UYOL :YJOX MAN ‘sisdpoupy vIvG JvI1403a10) (0661) ‘WV ‘LLSTYOV 

"E8T-PST ‘E ‘E ‘SOUSYDIS Q1u0UOTZT pun ssauisng fo joUsnor “SMO 
9910J-JOqe] Sso13 SuNeUINS| ($861) ‘VW ‘UANTIAZ }9 “f ‘AMOdV 


GIHdVaOOITala 


‘(p nes[qri 
9] JIOA) SqD PB] Op eae uoTe[Ndod ev] sp UONLdIJISse[D ap 
INIA, | PP WOTW SuoneUNse sop suplyea e| op ueOodut 
InajoeJ uN ang sed s[quies ou sUUaTAOYIeW osguJodAy | 
op d0Y99,| ‘Uouesnainay] ‘aseuJodAy 93399 ¥ dIUToNe JUDAS 
-nougs juayod jaued un .p UdAOUL Ne sal|[londel SaguUOp Sep 
anb ynod as TT ‘WO TW osaIeNs P| Op 919 P| Iso QUUDTAOYIEU 
esqyjodAy,'T ‘ajapoul np gypryea ey] JoIyugA Inod snssap-t9 
(S) 19 (Z) (1) saraq9 say aonb sanbiydde ang-ined yuounod 
aU saiskTeuR so] ‘Uanbgsuoo sed ‘jo joued sed saignbue 
Sa] suep saqruodsip sed juowonbidAy juos dU MAIAIOJUIOI 


op SeguUOp sof ‘sIOJaINOL, “WOT SUOTIeUITISS sap SIIpITeA 
P| Jon[eag Pp SIULIEd JUO MOIAIOIUIQI Op SaguuUOp sz ‘askTeuR 
o0U sued ‘suUOTIeNIS sone pe sduaprd deae sanbidde 
}10s apoyjsu ey onb suopueuWOsal snou ‘saanaidg sou suep 
sye][Nsg1 SUOg Op 9UUOP & YOTW alseiens ev] Is sway 
‘SdO B] ap soguuop sap syttenb 
RP] Op SopNig sonjny op suep SdgJOPISUOD JUSIOS sapoyleul 
xnep sof anb suopuew0dal snoN ‘A\-H epoyyeul ey op 
SUPITRA PT WOWOTeS9 SULITJUOD AA-H 19 WOW suoneumso 
So] BUS BOULPIONUOD 91J0J VY ‘SqdD eI ep sate uo 
-e[ndod e] ap suonewinse sop apnynoexe | ap uonen[eAg,p 
SPOYIPU oINV BUN SUIWIOD IdI9PISUOD IOS WOTW epou) 
-9UI e[ onb sUOp suOpueUTWODa SNON ‘siIpI]eA ap saanoida 
burs say Inod sje1[nsgi suog op guUOp & spoylsu VT “‘VWOTIN 
ISINENS LP] OP SIPI[eA k] INOp Us a19U Op UOSTeI sUNONe 
sosk[eue Sod SULP 9ANOI) SUOAL,U snouU ‘gUINS9I Uq 


(€ nes[qe) 
3] JIOA (9661 YUIMISED 39 IIejoUIS ‘Qg6] SJoWUINS 
39 BqINIOd “C861 JOUTISZ 19 PMOgY “Cg6l BNYD 
19 Joy[NJ o[duroxe Jed MOA) saInotigjue sapmg Pp Injoo 
DOAR PEPIONUOD SUOTILUIT}SO S90 Op INepuvIS op dIPIO || 
39 1o[duza-sues sap uonendod ej inod sgarasqo 919 1U0 
S9AdI9 snd say INdL9 .p xne} soy ‘snjd oq ‘uoTeInd01d 
Jed s}uepuodgs say nod aonb uoneinsoid sues s}uepuod 
-91 Sd] INOd S9AIASGO JUDIOS S9AI|P SUTOU INILII Pp xne} 
sop onb ue[NOA saiInoligjue sojus}e soy D9Ae JUDTep 
-109U09 aIyNe | & UOeINsoId suvs/ooae odnols uNn.p 
SUOTJVUINSA SIT “sa[qisne[d jus;quIas UOTIBOIJISSe[D 
op Inds .p saipiqeqoid sop WOTW suoneuinss so] 
UOIVIIJISSE[D IP ANIIA9,p spipoid sap opIqisne[gG  “¢ 


"SdD PE] OP 19 MSIAIOUIAI Op saguUOp 
ap Satquiasusd sa] Inod sdwia} asp spousd op saousl9jjIp 
Sap & a]qeNquye a9-jnod so AA-H 19 VOT Suoneul 
-1]82 S9] OIIUS SOUSIDJJIP BI “C661 INOd “9661 19 S661 
inod WOTW suoneuinss say 90Ae USIG JUSpIODUOD 
Inb dd1pul.| op suOoTeUINsS sop olInpold op youLIod 
‘saatjesed soinsou sap asayjodAy,| sed o8rxo,u mb 
‘M-H epouleul eT ‘sojayjered soinsous sap asgujodAy J 
ap 99499, Op IULI[NSa1 OUUOTIPeN IMsyeUlNjsa,.| Ip 
steiq of sed oryred ua ywonbi[dxa ,s s2oUsISJJIP $2 ‘9661 
19 G66 INOg ‘(9 Neg] qui 2] LOA) JHOVUI BLIOS9ILO eT 
Inod a11ej uoldaoxe ‘saguue sio1) sa] Inod ustq Zasse 
JUSPIONUOS SUOTILWINSS Pp So[quIssus XNEp SAT ‘SdD 
B] OP MIIAJOJUIal op oWUIeIZOId Np segin sayuepuod 
-SOLI09 S9}DAJIP SUOTILUIT]SO XNV BOUSIQYOOUL ,p SOIPUL | 
ap sogsl[gpoul WOTW suoneumnse soap giseduioo 
SUOAR SNON ‘39U919409 op anbuvul np }s9}01-}S9} 
adA} ap 32 SagSt[9POUl SUOIZEWIIJSS Saf 9.132 PADDY ‘YP 


‘SdO 
RB] OP MdIAIOWUL.p seguUOP se] saguue sonbjenb ep 
9p9ogid 1U0 MaIAJOJUIQI Op Saguuop sey onbsind ‘sdua} 
ap apousd ap seouslgyjip sop sed sonbrydxo,s juores 
-Inod sadudlajjIp sad “sojuepuodsal09 AA-H suoreU 
-1IS9 $3] 19 WOT SUONLUINSE Soy B11US SOATVOLJIUSIS 
STRU SO[QIe} SSOUSIOJJIP SAP PAIOSGO SUOAK SNOU ‘EGG 


@]U9}e] aINjonuys ep ouUaIAOMJeW asAjeuUe,| ep sypIeA :AUeysng je JeWelg OLL 


Inog (¢ NeIIqQuI I IIOA) aOUePIODUOD 9109 oUN 
JUDTEISOJIULU S qd B] OP SOQUUOP SI] 19 MOTAIOJUIOI Op 
seguuop szj Inod sdura} ap sapoued say sazjanbsay 1nod 
soguUe XNIP ‘9661 19 S66I INOd 9ArasqoO 919 B POD 
-98 U0g UN “WOT suoneunse xne 19-saT[ao aredui09 
SUOAR SNOU Jo asuOdaI ap sayT]Iqeqold sop UOTeUIT}SA | 
® M-H epoyieu ey anbiydde suoae snou ‘(9661 
‘C661 ‘661 JUeAR) sapotiod ston nod ¢q_ PI Op segJ 
-1] s9yo01ddel UOU MOIAIOJUIOI-MOIAIOUL.p SoguUOp 
Sop opie | VW ‘asuodga. ap saztiqeqoad sap 3s9}0.1-}89} 
ad} ap 39 So9st]apoOUr SUOIJEUIISA Sa] 91}Ua P1OIIW —“¢ 


"SagUUR SION] So] INOd INdT[IOUW 9] 
919 B UOTJBINDOId SUBS NO DAB B[GRURA PY] JURLIOdUIOD 
Q[Qpoul a] siveul ‘apnya.| SUBP Sd9IJLIQA 939 1U0 
juawiodnois ap sayqeuea sane p ‘snjd oq ‘(] nesqe) 
3] JOA) saguue sinaisnyd Ins juswa\sn{e | op sIIpITea 
OUN BYdJJe p [QPOW aT “saguur sto) sa] Inod yenbepe 
juowsjsnfe un Jo}sayIUeU YeIANpP [e1gUss s[epoul 
ouIguI af ‘sue aenb juepuad sajqeis yusIos SqO 
Re] Inod asuodg ap saqiqeqoid sap sinjonys ey aonb 
32 datOe UOTe[Ndod e| op onbrueuAp vy onb Jasoddns 
VY ‘uonejndod oswgui ey inod anne] & juepuod 
-9pUI UOT[HULYSE UN,p jUdUTaIsnfe UOS 4Se a[Qpou Np 
SIPI[LA PB] Op ANdJVOIpUl syne UA *Sd\ BI] 9p 2.13Nne,] 
¥ oQUUL UN p aJQpow Np juswajsnle | op aIpYeA —°Z 


“saguue SIOJ) say Inod 
soguuop xne yenbepe juowajsnfe un 1uINO} & B[Qpour 
99 ‘sougsowoy uou ssuodg1 ap say{iqeqoid sop 
}0 SolleUuUOTIe]S UOU Ja SoUgsOWOY UOU JUSUIAsULYD 
op saupiqeqoid sap ywestogid inb “(| neapqe} 9] IIOA) 7 
Q[QPOUW J] 919 & INA|[IOU J] ‘saTepOU ap B1I0B9}e9 97399 
Inog ‘uoneInsold sed sgusaul sno} sa{dA9 SION Jo UONeI 
-noold ed sguoul jUaWatNas safoA9 xnap ‘uoneindoid 
SUBS SQUSW JUSIUATNES Sa[DAO xnap ‘UoneINIOIJd sues 
. SQUIU SNO} S9[DA9 SION) :saLIOS9}e9 ayeNb uoTas (UOTRI 
-nooid sues no deae) asuodai ap adXj 9] Jed atuljep 
juswsdnols ap sjqeLieA uN syOdUWIOD saguUR SIO} $3] 
Inod juawajsnle InajjIow 9] 39 stuOWITOJed opuvsis snd 
RI JULISOJIUeW O[QpOW FY] “(UOTeINDOId sus NO ddA) 
asuodgi ap 3dA} 9] 19 MOIAIOJUI.P BpOU! 9] ‘aILIeTOOS 
RI ‘AXOS JI ‘d0vl LP] ‘a8",] UOTAS solUIJap jUsWIadnoIs 
op so[qeueA sop jueyodu0s WOT S2[2poul ap xtoys 
UN JNO} SUILWUILXS SUOAR SNON ‘SISI[9POUI SIsoUseIg “| 


INS DUIWIOD YO TW oiserens 

PR Op Supl[ea e] op suotsuowtp buto aspsioge suoAe snoN 
‘VOTW alsgiens be] 
OP SNPIeA P| JOUTLULXO & OpIe JUO SaTIo “WOT Suoneunss 
xne sagieduiod ‘¢ gd PB] ap UONLOIIJISse[O ap INdLIS,] op UOT) 
-PUINSO Pp d159}e1)S aIINe oUN gJUaSaIdaI JUD MOTAIOIUTOI OP 
soguuop se] ‘said nod e sdura} ap sapotied sauigui soy Inod 
SdO 8] 9p sooyooiddes uoU SMOIAIOUIOI Op SogUUOP sap 
oirejuowlgtddns askjeue ouN QUOU SUOAR SNOU 39 ‘966] 12 
C661 ‘€66] SogUUR SION) Sap oIISOUIL JomwUed aj Inod ¢q_ 
Rl OP MOIAIOIUI.P SoQUUOP Sap asATeUR SUOAR SNON ‘SdD 
R[ 9p SoguuOp sap gyyenb e] asp uoNeN[eAg,p SaTjouuONIpen 


691 


SOPOYIOU SOP ININSGNs sUILIOD YOTW BWSeaeNs Pk] op 119 
-POLJJO ,| JOUIWLIDIEP Op 19 ‘SqD PI ap sanoe uoNe[Ndod ve] ap 
UONVOIJISSE[D OP INDI] OPP WOT SUOHUINSO sop syIpPITeA 
RI JOUTWUeX Pp 9]U9} NOUNS ew sYdIOYIO1 aJUESoId eT 


SNOISNTONOD LA FAIVIWINOS = °S 


"R]-oouUe 93399 INOd sUOTeUII}SO Sa] OIJUD WRd9 | JUSU 
-o][oNed surow ne Janbrydxe yeunod ejaD p66] & Inoue 
MOLAIOJUIOI-MOIAIOJUL Pp SOQUUOP Sap a[qUIesUd | op UOTONI} 
-suod ev] Inod sagsi{NuNn sdura) op sapotied sayualayjIp soy sed 
SOTDINISQO JUOS RI-sQuUe 97999 INOod A\-H 39 YO suoneu 
-1]S9 So] AIUD sUOSTeIedUUOD Soy ‘¢ NeITqe} NP UOISsNosIp FI v 
910U 939 B [I SUIWIOD ‘sIOJaNOL, ‘¢66| Ue nod onbyenb juarey 
-JIP S]I,S OUIQUI “966 I 19 S66I Inod uaIq zasse JUSPIODUOD 
VOI 1° M-H Seoipur sat onb juowsfes9 Jojou y 


90°6 OLL 6E LE 66'S VOIN 
95°8 SOL Or 6E LOS M-H 
(1S‘0) (9S‘0) (89‘Z) (6€0)  s]JouUoNIpen 
19‘0I Soll LOSE €6'S uoneuns” 
9661 

TL'8 GE 61 9€ 90°9 VOIN 
L6 868 LE 78°9 M-H 
(€S‘0) (9S‘0) ($82) (pr‘0) — a[Jouuontpen 
tr‘0l 080! 879 69°9 uoneumnsy 
S661 

€L'8 €9'L ¥0'87 se9 VOIN 
8L 01 LOO! €6'VE [RE A-H 
(97'0) (L7‘0) (OID) (y7'0) =. a]JouuoNIpeN 
SOTI 966 6b'€€ 918 uoneunsy 
£661 

tojdwia 
jeqol3 JHoeUy pets nee | uoT}eUuINsS Pp 


so1puy «= SANNOV UONL[ndod ej ap uoNKoYIssejQ ~«—-—- @pouplayy 


aajoe uonelndod e] ap uoNeoryisseyo e] Jo sQUUR | 
UOTIS JOUSIZYOOU! Pp SdIPUT,| ap SaT[auUONIpeN 
12 M-H ‘VOIW suoneuinse sap uostesedui0p 


9 neaqey, 


Tp 
SO]JOULONIPeT) SUONPUII|SA Sap srerq ne ated usd sfqenquye 
DUOP IIQWIIS OG6I 19 S66 ANOd JoeUI sOZg}e9 eI Op 
SUOTJBWNSS SI] JUD DOUSIDFJIP 'T “%O] Op UoNVoIIUsIs op 
NVIAIU NV 966 12 SHG Nod ajafal a9 & 189} BD ‘aUOqI] Op 
Soizop 9 JURAR 9IB9-IYd UN eB oINaL9dnNs 1s9 sofapoul xnop 
Sa] Inod saour[quissiesa sap yoddes np sgues-1y9 say oUS 
SOUDIAAIP BI 18 agjafal Jso ae3a oyT[Iqely oun.p oseuodAy TJ 
#1 Py FlPy UONOLYSOI Be] Bp soUasqe, | Ua Ja JdUaSgId 
ud M-H 3dA} op o[apow un,p yuoursjsnfe | yueredui0s us 
JOIJLIVA J] Op IQIssod jsa [J “SMOTAJOJUI XNp Soy Inod afes9 
sed js9,U SINdLIO Sop UOTIyedaI RI IS SOJUSIATJIP soUpIqey 
SOP JUOIBLOAUIOS Sd LP] OP MITAIOJUI9I LP] JO MOTAIONUL TJ 
“QAa [2 don duOp 1S9_ SqD BP] SP BOUSIDYOOUI ,p 9dIP 
-UL,| ‘SdD PB] Bp SeguUOP Soap a9 SIT[IQeIy VI DWISO-snos 


0002 aiquwiec~ap ‘ajenbuse,p senbiuyoe | 


yS9JOI-]S9} DI[IGUIJ VP] Op JOUUONIPeN InaeUINS | ‘MOIAIO} 
-UI_ Pp SOQUUOP SAP I][99 B IINSLIQJUI SO MOTAIOUTOI Bp sogu 
-UOP Sop sitTIqeyy ey ‘ajduroxa sed ‘tg “agstelq yso soIpul | op 
J[[SUUOTIPeN UOUIN}SE | ‘SoJUSIOJJIP sayTIqely sop yuowod 
-W09 MOIAIOIUI9I IP 19 MOIAIOJUL.P snssadoid say] ts anb 
QUOU! & (C861) SIUA-SIeIq Sop SnsUdD dU} Jo NeaIng a] 
“ -sgjayyered sarnsour sop aseyjod 
-AY] OP SUPIBAUL | Op 1ULI[NSoI s][OUUOTIpey UONeUINSa ,_p 
I3IeNS Ll OP SIeIq UN eB B[QeNqlINe Isa sOUdIOTJIP 91109 
onb onbipur agssnod snyjd askyeue sug ‘ajuaie] oimonns 
9P SO9SI[PPOW! SUOTJLUITSS Sap a1jNe | No sun | aonb sapuvss 
snjd nod un quos 7 ap sat[auUONIpen suoTeUINsa Sa ‘966 | US 
19 C661 US JOU 911039}¥9 P| Op Iso Inb 99 Inod ‘stojoino 
“saguue SIOJ} Soy Inod sayjouUONIpen 19 AA-H suoTyeUINse 
SOT SOAR JUOUWITRI9UIS P1ODNe UOG Zasse Ud JUOS YOIW 
SUONPUWINSS Sop INspueIS op sIpsO | onb uarq Issne WOTNW 
suOneWIss sap [eigues [ljoid 9] ‘ajquiasua,, sueq 
*M-H suoneunsa say inod 
sodA}-sinolie sop uoneurxoidde ouruoo Jastynn ined uo J 
onb saffouuonipes suoneuinyse say Inod sadAj-sinale sap 
OISIX9 [I ‘SIOJAINO TL, ‘WUOWdT[IWIOJ asouJOdAy_,| JollIgA op 
g[qissod sed jsa_.u [I ‘SdIpul | 2p AA-H NO WOT suoneuinse 
$9] Inod sajqruodsip sed juos ou sodAj-sinaLa say onbsing 
‘QA9[9 INdLIO,p xne] UOS op osned zB Jol[NoMed 3919]UI 
UN JQAdI Io[duIa sues aLIOZE}eO eT “UeAvIedNe sWIUIOD 
‘sduia) op soporied ston say Inod souaigyooul ,.p sdIpul | 
OP UOTLUII}SS.P S9POYIOUW SIO] Sa] aUOW g NeIIqQu} J] 
“sapoyjeul 
SION] SOP SIPI[CA PL] SULITJUOD SUOTeUITISS SIO) soyT oud 
plosor J ‘(S86 Snsuad oy} Jo neaing “S'-) sayjouuonIpen 
SOPOUISW OP ople.| & SdD LI OP MOIAIQUIDI op soguUOp 
SOP JUSUIIJSIIIP 9NUD}GO SdIPUI | op UONUINSS,| B SUOTFUIN} 
-S9,.P SdTqWIasUd XNep Sad a}INsUS suOIedWOD snoN ‘asuodal 
ap souiqeqoid sap A\-H SUONeUINSS Sap apre | R SUISS ong 
jNdd SMOTAIOJUIDI-SMITAIOIUL Sop ONAdId UOTIIJISSEID P| 
Op neo][ qe} sWIgIxnap UL] “sAIOSgo IIeI9 NeITqQR} 9 IS QUIWUOD 
neg[qe} 99 & sdIpul,| Inod anuLoj e] ayInsua suonbrjdde 
snou jo ‘asuodal ap sayyiqeqoid sap YOY suonewn 
-S9 Sap Ied & SMOAIAIOJUIOI-SMOIAIOIUI Sop NAgId gSIOIO 
U1] op nes[qe} a] SUOWIT}Sa snou “UdWaT[aNUEssq ‘osuodgi op 
sil[iqeqold sap saest[apoul WOTW Suoneunss sop apre | 
e sdwia} op opoiued anbeys inod soua19yooul .p sdIpurl,| 
SUOUWINSS SNOU ‘[*¢ UOTDS PT B ILIOP 919 B [I OUIUIOD 


JIUZAIZYOIUL pP SIdIIPUI Sap UOSIeARdWIOD Zp 


"yINS 
inb uonses eB] suep Inapuojold us snjd sagIpnis JUOS S9OUSI 
“2JJ1P SPD “C661 ua.nb rginjd “gel & C61 OP Sel[[Ienoes 
919 1U0 P66 B SAINOLIQIUe MOIAIOJUIOI Op saguUOp 
so] onb yey ay sed ang-yned juanbiydxo,s 19-sap[ao ‘stoj 
-91N0} ‘satqetogidde saoualajjIp Sap JUSYIJe p66] & Sons 
-g1ue suoeUINse say ‘Iojduie-sues sap uoneindod ey] ap 
3s nb 99 nod AA-H 19 WOT Suoneunse so] oUe (%¢ ap 
NIAIU Ne) DANLOTJIUSIS JUOWONbYsHEIs VUSIIIJIP ouUNONe 
JUDISIJIULUL DU O66] 19 SHG SAQUUR So] ‘plOdv UO Zasse 
Ud JUOS SUONRUINSA,p Sa[quIasUe xNap so] ‘[eIgUas UY 


(‘Oo GD (‘0 © D (‘0 (*‘o) 
086 l6 786 O16 8'L6 8°96 Jnoeuy 
CHO Cy CMs KOKO GRO) XeS0) tojdura 
60 00 L‘0 50 SO eal sues 
CHORIG DEAT 0) GDE <0) Go) 


Ul 97 Ul Gg v1 O7 Aojdwg jnoeuy 
CD) ay (ES) (Gi) G9) (G:0) eGo) 
Oa “Cie Wow “ye Pini TAA Jnoeuy 
CAD) CRY ED GEO GAD Ue 1ojduio 
al, “SHS TOE GO I Ya surg 
OD) ESD) (GO) *Say EO OW tojdura 


9'8 o'r CieeSaLT [flo = wl YG 
(0) (mm) (10) (0) (10) § (€0) 
80 00 80 5‘0 60 L0 Jnoeuy 
(10) (‘0 (do) (@m) (1) (Oo) tojdura 
+0 70 50 00 €‘0 00 surg 
(lo) (do) do (Zo) (Oo) (€‘0) 
886 966 L86 S66 886 £66 
VOIN M-H VOIN A-H VWOIN A-H 
9661 6661 £661 


(sosaujuored omuo sodA}-sy1e99) 
o9uUR | UOTAS SD Be] ap asuodai ap sait{Iqeqoid sap 
M-H 29 VOI S2est[apoul suoneuinse p uosrereduioag 
g nesjqeL 


gXkojdurq sues 


gkojdwq Aojdurg 
apATSqO s199a 


uOneoIsse[D 


“M-H/VO'IW Uostereduios ve] suep 
Jorpnored 1Q19}UI UN IQAAI 19-a][99 ‘To;dura sues a11089}e9 
RI Inod nusains sd YOIW osaIeNs el op saaja snid 9] 
Inds .p xne} a] anbsing ‘saguue sto soy Inod uosteredui09 
Rl aNUOW NesIqQe} FY] *(6[) US PF SINSOUL PI SIIP-k-3S9 9 ‘aTeU 
-I31IO MOIAIOIUL | & SOQIOOSSE UONIIJISSLD op sai[iqeqoid 
S29] JUOS A\-H{ SUOTIVUITISS Sa] “«[eI0L» sagjyenbye Z neayq 
-0) NP SoUusI] sep sajjao aonb souigui say JUOS WOTW suon 
-RUINSd Sa] “S NeITqQe} dB] SUBP SPUINSEI JUOS M\-H 39 VOTIN 
SUOTBUINSA Sop uOsIeIedWIOd 37199 ap sjey[Nsol sa] 
“sogydoiddes UoU SMOIAIOUIOI 
9p UOTTHULYSE UN & 2d"I3 BRET B C86I OP Sel|[fondal 919 10 
P66I & Sainotigjue soguuop sea] ‘sduia) op sopotied sao ap 
SaTSOUIL] ayENb sap s[QUIdsUd,| Op MOIAIOJUIOI op SaguUOp 
SQ] Jast[NN nyyej ouop ke IT] ‘sazqeisul yQIn{d 939 UO suONeUT 
-1]S9 So] ‘SUOT[HULYDE Sop dI[[Ie} o[QieJ Bl Op osnvd v ‘JUOUIOS 
- naIneyjepy ‘sdwia} ap sapotied soo op jueWe[Nes aTsoUl 
-1) Joruod ne asAjeur ,| JOU] Op 91U9} SUOAR SNON “9661 
‘C661 ‘P66I URAL : VOT aisaiens anou op sopousd stony 
SQ] SOAR JURPIOUIOD sdula} ap sopousd ston inod agysoiddes 
UOU MOIAIOUIZI Op 19 MOIAIOUI.P soguUOp xne M-H 
g]Qpoul np juswaysnle | Inod gsi[NN 919 B JAA [919130] FT 
‘g[Qpour np juowaisnfe | JaIUI9A Inod gysql] sp gisep uNoNe 
JURSSIE] OU S Px PxD Neayqei np sajqiuodsip gusql op seis 
-9p ap siquiou ne puodsaxios inb a0 ‘aguue onbeyp Inod san 
-guiesed 9] aus A\-H ascjeur | ‘Isuly “YO TW alsarens | 
SULP S9ST[NN SIANOL, | OP SIA-R-SIA UONENIIS B] Sp SALIOS9Ie9 
SION & UONIIJISSe[D Be] SUOSATEUB SNOU 39 SadeI Sa] $3jNO} 
OP UOT[MULYDY ,P SAIQUIOU SOP SUOI9PISUOD snoU ‘asATeUR a1} 
-ou surg ‘tojduia sues a gAo[dwa saios91¥9 SOU ap SUWOS 
BI ISA BUOZgIed SIQIWEp 99D “JV 39 JNOVU! :9}ANIR.P 
SOLIOZ9IO XNap B 39 SOYDIL][G SOUT} Sap yo soULTg SOUWOY 


9]U9}e| GuNjoNsIs ep euUsIAOYIeW esfjeue,| ap syipIeA :AUeysng je JewWe!g 891 


Sop & ISATVUL INI] JTW] JUO YUIMISeD 19 ITepoUIS “osATeuR 
anou suep JuouIadnold op s[qeLURA 97199 JUSWUETeS9 SUOSTTNN 
SNOU 39 “JUSWIddNOIS Op I[QULIVA DUILUOD OX9S 9] VSI[IN JUO 
‘SdD BI op sane uoneindod ve] ap uONedIJISse[O op INL | 
op oshyeue Inoy suep ‘(Q661) YUIMISeH jo IIepoUIS 
‘2 ap sinofno} pued 
-op ‘°'"v arp-e-1s9,9 ‘Juowut ya 1ojdura sues ‘gkodure sati03 
-9189 SION] Sap souenbeyy ey aonb stpuey ‘5D ‘adnois ap ajqeu 
-PA e] op sed quopuodap ou «*!” 


<3 | 


a 30 *'’u aonb gsoddns juour 
-g[&39 1S9 [J] “SI[IqeIJHUSp! | op UONIpUOd ouTUOD 99}dope 
180 FOI esamsodAy,] “quoutaredas saquimso quos ‘*!"4 30 
*!?4 alip-R-189,9 ‘Saunsou xnap set Inod ssuodal ap saiTiqeqoid 
SQ] 10 SIINP9l ISO MOTAIOJUTI OP 39 MOIAIOJUI .P SosuOdaI So] 
Inod sajatjered sainsoul sap asaujodAy J ‘a[gpoul 9d sued 


x 
(61) 1 Py ly 3p By x _ P08, 


"JURAINS 9] 189 JOA 10 INF Jed gsodoid ’””*u inod 
J[Qpouwl oJ ‘nes[qe} np (, ‘v) s[NI[99 eB] SUep sesseId IOS 
3 odnosZ np queagjer ouuosiad oun, nb ayptqeqoad ey "a 
HOS *, PVD 910U MOIAIOUIOI XMOTAIOIUL xodnoIs nevsfqei Np 
DOUL[QUIASIVIA B] SUOIOPISUOD *(P) Ud SUTUIOD SIUTJOp JSUT 
odnois op a[qeuea oun ‘dD iOS “UOWdANIadSal ‘MIIAIOIUIOI 
OP 10 MIIAIOJUI,P SUOTIROIJISSeIO Soy $ W 30 KP Wos ‘guuop 
JUSUIOUL UN & S}AIIOV | OP SIA-B-SIA UOTJLNIIS ] OP I[9OI 
UOTLITJISSe[O BT ‘¥ JOS ‘snssop-10 s9uUOTUSW UOTIe}OU 
PR] SUOSTTN SNON “(OR6T) JOue AA 30 INH Jed gsodoid psoqge .p 
Q]Ud}X] GINJONNs op sepoul op od} uN.p opre.| & SdD 
R] OP MOIAIOUIAI op soguUOp sep ssuodal ap sai[Iqeqoid 
Sop SUOTFeUIT}SS Pp s[quIZsUS aI]Ne UN ITU9}GO Jned UO 


JIVAA-ING ep uoynewMysy ['p 
AaqHOOUdd Va NON 


MGTAUAINIGY Ad LA VOTN AdAL Ad 
SNOILVWILSH SUG NOSIVUVdINOD ‘+ 


Z=XIT=9 


u — 
066 686 886 L86 986 (992 «aNNe» jane») Ig 
Lior ded re 
008 €6L LBL I8L 9LL (1291 «UIQYd» |«WQYo») Ig 
(AOWeW)) 
OT=V% 80=Y% 90=% PO=Y% TO=Y 


(399109) Ig 
v=V='¥ 
(sasejusoinod sap 1UOs saN]Je0 ap saejUS Say) 
SUITAOYICU UOU S}USUTOsUeYD 
sop Inod 93991109 UOTeOTJISse]O op suoNneUINs| 


p nea[qeL 


‘SdO PI op uoNRs 
-IJISSB[D OP INDIA .p Say{Iqeqoid sap UOTFeUI|SS,| SUBP STeIG 
op sjuepOduII doINOs duUN Ng sed a[qUds dU SUUDIAOYIeU 
sseujodky,| op supyea op onbuew 2 onb juonbipur 
sapnig xnep sa] ‘SdD eI ep seouuop sap nod YOIW 
SI[QPOU SOP 9Ssa}snNqOl op 319}0BIVD SUIQUI 9{ JONSNIII,p 
ule aJUSISIJIP nod un a139}e.Ns ouN gstjN yuo Inb (6661) 
JosJoq[opury 39 Aroysng op xnoo e yuapuodsaLi09 p neayq 
-b] Np sjey[Nsai sa] ‘adevjusoInod ap sjutod ¢ steuref assedap 


Z9l 


OU USIAOYICU UOU JUSOSULYD op sazT[Iqeqod sap eB a[qenq 
-LJe NOSge sieIq 9] ‘sulOWTURaN °7‘C ‘sesestAUD sIQIeJ sntd 
B] INgpVA LT B JUepUOdsaLIOD [RWIIXeW SIRI 9] ‘JIeSOU STeIG 
un jUa}sayIuRU SUOeUIN)Sa sat “y ap 19 'y ap saimed snd 
SINIJVA Sap INOg “NAVId SUIWUOD JUSUIDJDeX9 JUOS 93991109 
asuodal op SoAeUINss soyIqeqold so] ‘UdIAOYIeU BfQpOUl 
un e puodsaios tnb ‘o‘{ = “y = 'y snod onb sa}0u y ‘Z‘9 ep 
sratjed sed 9‘ 12 7‘9 aNUe Y JoUeA Wey UO,| No Y= “Y= 'Y 
Inod uone[nulls ev] op s}eI[Nse1 Soy oUINSEI p NeITqQe} VT] 
“UONR] 
-nulis oun, p uoNNd9xa anbeyp nod sagstogid “y apie "y ap 
SING[LA OP OpIe | & (BT) OP 39 (LT) op AMued kv sginoyed 919 IUO 
sInoye1guINU Sat onb sIpur} ‘966 [-€66] SESUIQUIOD saguUUOp 
sop sued B SopUIULIaI9p sINeTeA sINg] B SexIy 919 UO “y ap19 
ly ap sinoyeurwousp s3I ‘snd aq ‘suoTETNUIIS Sat so]No} suep 
660 & «|e "at ‘ay001109 «anne» asuodal oun,p giiqeqoid 
ey 10 0g‘o & tt!" ‘ayoauI09 «10jduia sues» asuodas 
oUN .p a]Iqeqoid eI axIJ SUOAR SNOU ‘9661-€66I SeQUUOP 
SO] SOAR 9ITWOJIUN,p 1onos Jeg ‘| 19. Q anus “y 10 'y que 
-CA UD DUUDIAOYW VseuJOdAY | Op 99Yd9,] S[NWIs sUOCAR 
SnoNn ‘| = “Y= 'y onbsioy sarteuuoneys uoU JUsWIAaZURYD op 
souiqeqoid & WOT 2[2poul un ze JUsUIA}Q[duI09 s1puodsal 
-109 B UOSeJ OP sayINpod 939 JUO Sag [NUTS sagUUOp sa] 
“SUOTAOYILUL JUOS SIVUI JO JOLIAQJ 91JUS VBLUIQYS op s}USW 
-oB3uryo say ‘{ = “y= 'y anbsioy ‘anb saj0u y “[s“ysQ anb 
suosoddns snou ‘ure 19 JoPJOxY IeAINs ‘stoy ouN oI09Uq 
‘Jotauel Ud «IO[GWId sULS» Jo IOLADJ Ud «aTNe» 939 IIOAR 
sgide ‘«tojdwia sues» a9g,p suiqeqoid ey e JUSUIOATLIOI 
‘sjuopgoeid sIoul xnap Sep SINOD ne «aIyNe» 939 IOAR sgide 
‘sIeUI Ud «IO[]dWId sues» 10s sUUOSIEd suN,nb sy]Iqeqoid 
ep yso “y ‘auwigut oq “[5'ysQ anb suosoddns snou ‘ajuow 
-3ne 1o;dura sues suonenyis op oiquiou aj onb oinsowl & 
a}UdWIZNe Io; dura sues sInowWiep ouuosied oun ,nb ayypiqeqoid 
eB, aonb syuour yuo Inb ‘(Qg6]) ule 10 JopIOyV Jed snus} 
-qO S}e][NS9I SI] DOA 9ITWLIOJUOD UY “s}Uapgogid stow xnep 
Sop SINOD Ne «1o[dus sues» 919 JIOAR Sede seul Ud «Iojdwa 
SUBS» 31}9,P SI[IQeqold Pv] & JUSWIOANPIAI JorAuel Ud «ane» 
19 JOLIAQ] Ua «IO, durs sues» 939 IIOAR Saude ‘sreul Ud «Io,|dwa 
sues» 10S suuosiod oun,nb syyiqeqoid ey iso 'y ‘tsury 
(SOON S32) as 


uw 
: anc, 
(81) CR Cee aa 2 On 
i) 
T4T=x|1=2), 
i 
(LI) ba Mice Sey > x 


No ‘UONe[NUIIs eT] SUBp yUeLI 
-BA ely yo 'y ‘sanguresed xnop luljap sJoye suOAR SNON 
"9661 3° S661 ‘E661 OP Se9UUOP ap sgUIquIOD sayquIesUA soy 
dOAv JURPIOOUOS oUIUOD sagstogid g19 UO a= “ly = *!Py 
asuoda: ap samiqeqoid sop yo “Mu yo Ow 
souaie] soyiqeqoid sj ‘uone[nuls eT op ona ud uon 
-etndod aun sqei9,p UI *D 19 g ‘YW SegAJesqo suonents 
Sg] Inod sa[qe[quias suOIIULJep Sop oak 7 = 7 NO _{ ‘y 910U 
‘(gnoeur no gfojdura) ane ya ‘| = 7 no J ‘v 910U ‘aseUIOYD 
:QUUOp JUSWIOUI UN & SUCHEN NO Sasse[D Xnap yUuaUANbrun 
JUeLOdWIOD sa}Ud}eT SOINIONNS sop SJgpIsUOD sUOAR snoU 


000z aquiadap ‘a}enbue,p senbiuyoe | 


‘UONRINUIIS B] 9p dIpPed 9 JURTJI[dUIIs UD ynO} BYdIOYOEI PT 
J9]UDU0 .p UIP “S9ITUWT] UONL[NUIIs op apnye ouN gUOU SUOCAR 
snou ‘uoneordde ajuesgid ey Inod (Z) ue suUaTAOYIeU asa} 
-oddy | @ SUOTIOLIJUT SOP 19JJ9,] JOIpNy9 JtoAnod ap uy 
“UONLIIJISS[D OP INSI,| OP WOT SuUONeUINsa soy 
JUSOUSNIJU! SANIV | OP SIA-R-SIA UOTJLNIIS B] OP SUSTAOYIEUI 
uoU sJUsWOSURYS So] JUSUMUOD NOARS op uONsenb xy 
B o[[olUed surow ne ssuodgl ouN IUINOJ B SUOYOIEYD snou 
‘uOl}des oJUSSoId Be] SUep ‘sUIOWURaN “OATIOR UONe[Ndod vy] 
Ins sajgnbus sone .p 19 gD ke] Inod ajureljus ang,p onbsi 
(Z) us sgwitidxe suustAoyseu asayjoddAy | ofjanbe] suep 
SINSOUL L] V JUDUISANLII SNSIQUIL 39 SJUSINYOOUT JUSUIATeS9 
JUOS UOTLJUSUINIOP LI Sp SoM) syeI[Nsal Sa] “aI9UEZOININY ,| 
@ OP SIeviq np 39 UOT[NULYSE | op o8vIN op steiq np aiduio9 
jus) UO, anbsIO] dWANIV.] OP SIA-R-SIA JUSUTAOdUIOD 
INjNJ 9] INs joJJo UN JUO BIN INg] NO s8eUIQYO ap soINeti9} 
-ue suoneMs so] onb UONeOIpUI oUNONe 9ANOI) U0 ,U (O86) 
seliog 39 ULUIYDOPY ‘SallepUOdeS SapNieg SINI] SUTLLLIA) JURA 
SOUWOY Pp opnig ouN suep ‘sIojajno], ‘suoNENiIs sao ap 
d9inp ev] ep onb ouiguI op aseUIQYD ap saInoLigjue suonenis 
Op sIquiOU Np pusdeap sseWIQYD INjNJ UN,p a1] Iqeqoid ey onb 
ronbiput nd uo (9861) Ure] 19 JOLOAY “SdO eB] ep Seguuop 
op apie | & ‘Isuly “uOTeJUDWINDOP ke] op sjuaI9YyoO nod 


$}8I[NS9I sap 9}dwI09 ag-jned yUapUsI UOISNJUOD ap sdozNOS 
S8D “egAIasqO UOU 9II9UNZOIDI9Y | Op 19 asuOdgI ap sinal 
-I9 Sap ‘UONDEI9S 9p STIG Sap osNed & IIIOIJIP Zasse so UOT) 
-sonb 91399 ap epnig,| onb anbiput quo sefiog 10 uewydey 
(E661 APIOD “6861 YOUAT ‘O86 seliog 10 UeUTYADOH ‘0861 
ule 39 JOLJoxY e[duwoxe sed 110A) topdua | ap SIA-R-SIA oN) 
-Nj UONENIS I Ins 1O[dUWIS | Sp SIA-B-SIA OINLIQIUR 19 dTTON) 
8 UOHLNIS L] OP J9JJO,| QUIWIEX JUO SINayNe sINaISN{g 
QHAIZIE | IP SIA-B-STA 
UOHENZS Ll Ip SUSTAOYALU UOU SyUsUTESUEYD 
Sap piesa] & WOTIN 21893838 eB] ap assajsnqoy 9°¢ 


“UOTIeOLy 
-ISS[D Op INdLO.| Ip SUOTeUINSS so] Ins soUeNTUI sq 
-IeJ oUN JUSWUITQeqold do1OXo DUUDIAOYIeU aseuJodAy,| ap 
d0Y99,J onb auRAINs UONOes eB] B SUONbIpUT snou ‘stoJaInoT, 
“QINSOU OUTL}IIO SUN SULP SoTqRIeA JUOS sUOTIeOI[dxa xnap 
sot aonb suowilyso sno ‘ouUaTAOyIeUL asguiodAy,] op sup 
-I[@A op onbueul np asned & seq 9] SIJOA STeIq UN yUdLTOdUIOD 
VO'TINW suoneunss say aonb so uoNeor[dxe anne ou) “MOIA 
-IDJUI9I OP 39 MIIAIOJUL,P SINALIA So] BUS sUOTIeIgION ap 
asneod & INvY JJ SIOA SIeIg UN JUd\sasIURUL SaATIeIedUIOD suOT] 
-PUINSd saz anb yso soUaIaIJIP 9100 ap uOTVoT]dxe ouQ 


(9661) 00°86 
(SIOW Z) SI‘L6 (S661) 0786 
1766 786 (stoul [) 1726 (€661) +8°L6 Jnoeuy 
(9661) L8‘0O 
(slow Z) p7‘I (S661) 69'0 
67.0 79‘0 (sloul 1)6I‘T  (€661) SL‘O toldwo sues 
(9661) €I'l 
(stowZ) 19°t (S661) IT‘I 
‘0 oll (siom 1)09'T (€661) IPV‘I phodurg Jnoeuy 
(9661) OO‘LI 
(stow Z) 91°8 (S661) ¥0‘9T 
€S‘6 or‘ II (stow 1) 178  (€661) EI‘II jnoeuy 
(9661) Zp 
(stow Z) €7'88 (S661) 60°9L 
LS‘88 9L‘r8 (stoul [) £788  (€661) 18°18 tolduis sues 
(9661) LS‘8 
(sioulZ) IS‘€ (S661) 98'L tojduia 
161 8L‘¢ (stow [)Z7S‘€ (€661) 90°L phojduig sues 
(9661) 6L‘0 
(stoulZ) 10‘ (S661) 8Z‘0 
€0'I CEL (stow [)ZO'l  (€661) 68°0 Jnoeuy 
(9661) LE€‘O 
(stoulZ) y¢0 = (S661) +60 
610 7S‘0 (stow 1)Z¢0  (€661) EO to[duia sues 
(9661) €L‘86 
(stow! Z) $9‘86 (S661) €L‘86 
8L‘86 rL‘L6 (stow 1) 99'86  (€661) LL'86 9 :Xoidwq = gAoyduig 
Be ssa TON 
MAIAINUIZY (1861 2p seguUop) (786 oP SepUUOP) 
SdO SIOWIUWINS 19 &QI9N}Od Joy[N{ 39 enyD VIIN UONLIITJISse[D, 


juoWaINoUgIUR sagt{qnd suoneUINse xne WOT suoneurnsa sap uosteredui0} 
¢ nesfqeL 


ejUa}e] siNjonujs ep euUsIAOWeW asAjeue,| Bp eypIeA :AU@ysng je J8WesIg 


991 


(I1‘0) COR CE0) ano) aa iD) (060) (11‘0) (110) (11‘0) 
00°86 0786 v8'L6 = OO'LT 70°91 ell 6L‘0 8L‘0 680 [eIOL 
(EOL MGGCOMEECONY CGO 8a) (€8‘T) (Z1‘0) (O1‘0) (€1‘0) 
ZS‘L6 9S‘L6 6v'L6 L0‘0@ IU‘ST Ie‘€1 660 780 a) uoneinsoid eg 
(ovo) (geo) (Seo) Ole) (8st) (L6‘1) (ST‘0) (ST‘0) (10) ynouns 
96°96 C186 7S‘L6 80°LI 6061 L7TI L8‘0 06'0 10'T uoneimood reg 
(LE‘0) (9¢0) (SEO) (6?) (¥6°7) (€8‘T) (910) (S10) (€1‘0) jnoyns 
8786 0S‘°L6 v7L6 §= OT FI £907 176 98°0 68°0 180 uoneinooid sues 
CON CACO)! © CAL) {Cire (AD) (6€T) (60‘0) (80°0) (OL‘0) 
89°86 86°86 Lv'86 9S‘rI 78‘El >8°6 cs‘0 €9'0 LL‘0 uonemnooid sueg jnoeuy 
CHE GAEO? — GREO) — Oke). Miczqy (060) (110) (110) (11‘0) 
L830 690 cL‘0 trp 60°9L 18°18 L€e‘0 60 ve [eI0L 
(070) (L‘0) (610) (62) (81°Z) (61‘@) (60‘0) (60°0) (O10) 
rom 760 660 ETL TULL 60°08 6£0 6s‘0 vE‘0 uoneinooid reg 
(970) (8z0) = (@Z0) OLE) (LE‘€) (SS‘Z) (€1‘0) (ST‘0) (110) ynoyns 
€S'T 060 180 PLZ 60°89 vS‘OL LE‘ ~S‘0 LE‘0 uoneinoold reg 
(87‘0) (170) (€70) (IL‘€) (Or‘€) (8r°7) (S1‘0) (€1‘0) (Z1‘0) ynoyns 
6L‘0 160 €0'I 09'EL ILL 89°78 870 0s‘0 870 uoneinooid sues 
(Z1‘0) Colo) (ro) (ret §=(r0°@) (Z9'T) (60°0) (80‘0) (80°0) 
or‘o ce‘0 8r'0 99‘6L ZS‘08 76'S8 Ir‘0 6£°0 €€'0 uoneinooid sueg — toy dua sueg 
(11‘0) (¢ 0 0) eee @ 9 0) (0000 ©) (060) (OL‘0) (110) (110) (110) 
Sle LUT Iv‘l LS‘8 98'L 90°L €8'86 €L'86 LL‘86 [ROL 
(810) (ZI0) (20) (Z9'I) = (rP‘t) (€v‘T) (STO) (€1‘0) (910) 
O€'T OST 9ST 0€'8 IL: 09°9 99°86 £986 99°86 uoneinooid reg 
(O€‘0) (SEO) LEO) ee (EGO) LOG) (96'1) (070) (70) (810) jnoyns 
Os‘T 660 89'T Sl‘Or 98°71 6L'II 9L‘86 9S‘86 79‘86 uoneinooid reg 
(¢7'0) (0€0) (470) (96%) (OS‘Z) (68‘1) (7'0) (170) (81‘O) ynoyns 
€6°0 6S‘I ELT ara 99'L 118 9886 19°86 16°86 uoneinooid sueg 
(€1‘0) COLO (GEO CS) ae (Soap) (960) (€1‘0) (Z1‘0) (Z1‘0) 
760 L9‘0 COT BLS 99'S ong €0°66 86°86 06'86 uoneinooid sueg ghojdurgq 
9661 6661 £661 9661 6661 £661 9661 S661 €661 adnoip 
JOU! JUSWIOT[90y 1o[dula sues JUSUIZTIO9Y go dura JUSUIST[Z9y ZAINSGO 


(sasayjuered a1jua sognbiput yuos sodA}-sinaa sa7q) 
d9uUR | 19 BdNOIS J] UOTAS 9}IANIL | SP SIA-B-SIA UONLMIIS LE] OP SIATLUIN|SO UONPOIJISSLID Op SaIT[Iqeqolg 


Z neaqeL 


“SOATeIedUIOS SUONRWINSS xne JUSWA}ION9 snd 
a] puodsauoo nb sduia) ap epoued ey] ‘¢66] Inod sajuep 
-uodsal10d WOT suonewnse sat anb sagaaye snd ‘adnoi3 
99 Inodg ‘lojduia sues jUdWIaT[991 souUOSIed sap uONe[nd 
-od e] Inod agaiasqo 3s9 BoUaIaTJIp apuvss snjd ey ‘samnaii9} 
-UL SUONPLUIN}S9 Sd] SOA OPIONUOD JINNL | B SAIL Pp B03} 
-89 UN .P YOTW SUONRUII}S Sap SATILIAI NILA PI ‘TeIQUaS 
UT ‘SdO B] OP MOIAIOJUI,p saguUOp sap INed eB JUDUIO}O0IIP 
sogiedgid yUOS WOT suoneuinse say onb sipue) ‘MartAro} 
-UI9J1 Op SOQUUOP Sap INS JUSPUOJ 9s SUOTLUINSS Pp Se_quIasUS 
SIOIWIIP SION So] ‘SIOJ oUN BIODU *“Sqd k] ap seeyooidde 
UOU SMOIAIOUIOI Op sWIWIeIZOId np 19 (S66) SJoWUINS 
19 BGINOd OP (L861) JAIN 39 eNYD sp suoneumsa 
xne sogieduiod jUOS saguU SIOJ] Sap suNdeYD InNod YOTW 
SUOTILUINSS So] ‘¢ NeIIGQe] NY "UONeUSWINDOp LI ap sedi 
S8[Qe[qUIOS SUOTILUINSS Sop & SqD PI OP UONPOIJISse[O op 
Soil[Iqeqold sap suoTeUISa SOU a}INSUa SUOIedUOD SNON 
“INdI9 .p sadA} xnap se] 
Inod 9Ad[9 1S9 XN} 9] SIPUL ‘SOATOBUI OUTUIOD sauUOSIad sao 
OP QUOLID UOTLOIJISSe[D OUN JSo Lo[dura-sues sap UONRIIJIS 
-SP[D OP INILII,| OP UOIIAUD SJOI] xNap ‘saguue SION sat Inod 
‘guusXoul ug *sagXol dure souuossed sap surui09 sauuosiad 
S99 BP 9QUOLE UONROIJISsEIO ve] aonb savsZ snd awetqoid un 


SOL 


1S9 SOAIDVUI SoUUOSIad sap SUIWUOD IO]due sues souUOsIod 
sop s9UOLID UOTVeOTJISse[O eT onb onbipul nesjqe) a7 
"$}e][NSOI $99 JUSpUd}-snos INb sasned sa] sIpueidur0s 
Inod snsusp oy) JO neaing ne jUsWaT[ON}OR JUSAINSINod 
QS SOYIIYIOI Sq] “SIOJ BI B XNap sayz no ‘uoNeindod ey] ap 
S[99J SJUSWUAZULYD Sep ajQ-jned 9}Q1JO1 NO ‘UoWIOTURUAI Np 
aouenbgsuos oun a4j9g-ynad 389 33}J9 19D) “(1000 > @ Anayea) 
jusweansedsal “9661 19 S661 nod %p‘pL & 10 YI‘OL 
B JUSUISATVIVAWIOD ‘4% 9‘[ 8 Op 11k}9 93901109 asuOdgI ouN Pp 
syiqeqoid ve ‘E66 UA ‘(QUOWOTURUIOI 3] JURAINS soguUe 
S21) 9661 39 S66] Inod aonb (inofew jUsWIOTURUIAI Ne oINSTI 
-9]Ue a9UUR) £66 Inod agazIP sn{d Iso UONeIE[DEp op apni} 
-9exo,| anb juonbipul syey[nsai $90 ‘slojaynoy, ‘uoTyeIe[IEp 
Op opnjdexa | JOIOISUWIe Pp Ue PEG] Ud gIURUIOI 919 & SGD 
Rl OP MIIAIOJUI Pp sIteuUOTsanb a] anb eIpUdTANOS ds UG “aN 
-n& | B SAQUUR SION Sap oUN ,p IO[dUWIS sus JUSUIT[I9I SOUUOS 
-Jad sa] Inod uonesepsep ap opnynoexo .p soualajip ey puaid 
onb uonosllp eb] Iso Z Nesfqe) Np jUvUdIdINs IeI[NSoI UA, 
“SANVOTJIUSIS JUoWIONbY 
-Stje}s Sed jso,U SOUDIOFJIP PT Stew ‘AxOIg APSO] Sdnols 9] 
anb 93901109 asuodal ap sit[Iqeqoid ay10j snjd oun eB aouep 
-U9} SUN JUSWETeS9 IsaJIUeUL J[ag APSO] OdNoI3 FT “(9661 
12 €66[ Inod saNeoIsIUsIs yUOWIONbNSNes) 91991109 asuod 
-91 0UN Pp dy]IGeqold ay0J snjd ey soOdwIOD JJag adnoisd 9] 


000Z aiqwasep ‘ajanbue,p senbiuyoe | 


‘NAQId SUIWOD “UOIIAUS 298 10 UOIIAUS %gYO aS aNe | v 
odnois un,p 39 orne,| & sguUUR oUN.p BURA 9190100 asuodal 
oun.p su[Iqeqoid x] ‘tojdura sues juowesl[ae1 souuosied 
S9J Inod ‘slojajnoy, ‘syuoevur say inod %/6 10 sgXojdwio 
So] Inod % 86 SUIOW Ne :a9gAIT9 ZassE Sd 91901100 asuOdgI 
oUN ,P sI]IQeqold eI ‘soATIeUL JUDWAT[99I SoT[90 Ja sagkord 
-WId JUSWI9T[99I1 SoUUOSIOd Sa] INOg ‘JNOeUT 39 Io[dura sues 
‘gho[ dua so110s9}e9 sot Inod p sJapoul np seein esuodal 
ap saiiqeqoid sap suoNeUtsa sa] aUOU! Z Nes{qei aT 


UOHVIYJISSE[D IP INI119,| 9p UOHLUIS| S°¢ 


‘aatoe uoe[ndod x ap UoNRoIJISse[o op 
INL | OP SUOTVUINSO so] sIINpold Inod p sJepoul a] DuOp 
SUOST[IIN SNON ‘a[QeTBA SQI} B[Qpow op jusUa}snfe un anbip 
-uI Inb 39 ‘a%¢‘Q op 3so BJQpow a] Inod ‘p opnyrUIssip op 
SOIPULT “(8861 SOOW 19 1661 YsIeURaYoIIN.C e[durexe 
Jed 0A) ajgnbua.p sepoyjsul say Ins settqnd uonejusul 
-NdOp P| & puddsaI0d ‘jIp SUOAR | SNOU SUILUOD ‘VAD “AXO1g 
/J[2S ednoid os] UOCTES SLIVA UOTROIJISsejO op Ina, onb 
gnjsod [1 ,nbsind sasuodai sap aogy} e] op ana ap jutod np 
J[qisne[d jUDUIITeS9 1S9 p B[QPOU J] “AI9PISUOd 3sa d sinayeA 
Sop d19}119 af onbsioy, apqeidaooe yuowaysn{e un IMuINOJ 
B [Ns J] 1S9 p BJQpou 9] ‘saguuOp op sguue onbeyd Inog 
‘(L66[ JUNULIA A) soguUOp xne saisnfe UaIq JUR}9 SUTLIOD 
(%S¢ ap BEpow! ap naa) CQ‘O > Pp JUeLOdUOD sayapowl sap 
SIQPIsuod UO ‘[eI9Uas UY ‘sisnfe yUDWIO}IvJIed JOS s[Qpou 
gf onb inod gn{[99 ap Jasueyo ke JUsIeIne Inb suoTyeAJOsgo 
sop uorodoid ve] sa (p) apNiTWISsIp op sorpul | ‘UIyUq 


"9]U9]e] 
QNJINIS Vp Sefgpoul sap Inod s139}eNs 93199 ap synosIp yuo 
(L661) Uorkeq 19 NYT ‘otuOUTOFed FP] & 19 JUOWIaIsNfe | ap 
SUPILA LT & USWA LISI IN|[Iow! J] SWIWIOD DT g e[qrey snyd 
9] JUeLIOdUIOD s[QPOUI 9] SUOJEPISUOD snoU ‘saTquIOARy JUOS 
OIA Np singjea sayned ap onbsing *(/p) a[apoul np stuour 
-tored P| 19 (77) 9[Qpow! np yuswajsnfe | aU stu01dWI0D 9] 
JUSWa[ONUassa SUINSSI OI FJ] ‘a[Qpoul np svoqt op seiZap 
sa] ajuasgidal Jp 19 [e10} UOT[HURYDY,| Op aT][Ie1 BI] ISA Ay NO 


JPN 801) - 77 = O14 


‘JINS SUIUUOD IUIJEp 9s nes[qei NP OI g sINsoul vy] ‘ajqeideo 
-08 JULI9 SUIUIOD [99 onb ayN0d Issne d Inojea ouN suOIJEp 
-ISUOD SHON ‘o[QpoOuw Np JusW}sn{eins UN JouTeNUO yeLINOd 
opuvis Issne d Ino[ea ouUN JOBIXa.p 1k] a] ‘asAyTeue aOU 
OP SUOT[NULYIE sop 39AII9 III} B] BP osnevo v ‘sIOJaINOT, 
‘yenbope s[apowl op juswiajsnfe un ,p jemiqey a1QI1I9 9] Isa 
C0‘0 & sInotedns no afe39 d Inajea oUL) ‘a]URpUodsaLIOD d 
Inayea ek] onbrpul ¢ suuojo9 ey] aonb sipuei “(gp ‘O66 NSASV 
MOA) , 7 seoue[quisstesA sap Wodde np sjfemiqey eLed-1Y9 
onbysiye3s ey onbipul neayqe) np p auUO[OD wT ‘soguue SION 
sap 39 sajapour burs sap afquiasuad | Inod aseq op jusUId} 
-snfe.p onbysneis ve] suonbiput snou ‘{ nesjqe} a] sueq 


Ak 


2G aia uae oa |x dy eG iz o‘q'p'd, 


1INS SUTWIOD 
aILI99 s nad (fp) ‘sayureNUOD S90 ap NEA u_ ‘d soy sno} Inod 


‘2]Qpow Np saneuresed ap s1quiou 9] ajUesg1doi «edu» onb J9}0U Y | 


sougsouloy uou asuodgi 


EON 1} 1960/0 coe - ve ap sait[iqeqoid jo salreuUONe)s UOU Jo SoUgsOWOY UOU SJUSUASULYD ‘p 2[2QPOW 
S00‘; = H9E- 0000 9°78 69 (4 SoJTBUUONE]s UOU Ja SAUQZOWOY UOU sJUdUAZURYD ‘¢ 2JQPOW 
O10; = EvS- 000°0 6SI Iv 99 SaJTEUUONR}S UOU SJUDUIOSULYD *Z B[QPOW 
vrO'0 80€- 000°0 SBS €Z 8 sougsouroy uOU s}UsUIaZURYZ_ ‘| 22PoW 
Sv0‘O0 = STZE- 000‘0 ZE9 LI 06 asuodgi ap sait{iqeqoid 39 sorreuuoTe)s 39 SoUgdOWIOY s}USUIasULY_ ‘aseq 9p 2[QPOJ| 

p fe): Ce) 2, eg edu fp 9661 ap saguU0g 


7000 = PET- OIr'0 Sc £8 v7 


S00';0 7ZLE- + 0000 78 9 rh 4 
8000 L9S- 0000 91 Ip 99 
€p00 Orz- 0000 899 € 8 
TROON SEC 000.0!) FL69 PLT 06 
P Old = ange gT yedu fp 
700 v€%- 1050 £€ €8 v@ 
S00; 98€- 9100 = #9 S9 ty 
4000 609° 9000 66 Iv 99 
LP0'0 692-0000», ZED £2 $8 
8r0'0 Oze- +0000 2S Sts 06 


sougzouroy uou asuoda1 


ap saitjiqeqoid 39 sarreuuone}s uOU Jo SoUQsOWIOY UOU sjUsUISZURYD “yp 2[QPOP 


SAITBUUONE}S UOU }9 SOUQSOWOY UOU SJUOUTasULYD ‘¢ I[QPOW] 
SalTeuUONr)s UOU SJUSWIASURY-_ °Z 9[2POW] 


sougsouroy UOU s}UaWIasUeYD ‘| 2]2PO] 


asuodal ap saitjiqeqoid 39 sasreuuoNe)s Ja sougZOWOY sjUsWIAasULYD :aseq op e[QPO| 


S661 2p SepuU0d 


sougsouroy uou asuoday 


ap saitfiqeqoid jo sosreuUONE)s uOU Jo SoUgsOWOY LOU s}UaUIAdULYD :p 2]QPOW 


sarreuUONe}s UOU }9 SOUQSOUIOY UOU S}USUIEsULYD °¢ 2[2POW 
saIeuUUONE}s UOU sJUsUINsULYD_ ‘7 3[2POW 


sougsouroy UOU S}USWIASULYD *] 3[2POW] 


asuodai op saitiqeqoid 39 sarreuuoneys Jo sougsoul0y sjusuasueYZ :aseq ap 2[2poy| 


Parla taigniee gi Rick ge ee ee eee 


Pp Oa = Tanoyea Aa edu JSP 


£661 ep Saguu0d 


OQUUR | UOJES WOT S2[gpour sazyne .p inod sgstjgpoul sonsouseiq 
[ neaiqeL 


aUE}e] aINjonys ap auUsIAOeW asAjeUe,| Bp sypITeA :Aeysng jo Jowelg POL 


(91) TEN panne dx|2,, 


SOP JULASIXA SOUTeIUOD So] JURSINPaI UD ¢ B[QpOW Np ANed 
B J[QPOU 9d NUA}GO sUOAR SNON “}[9S/Axo1g sodnois aenb 
sop one | & UN,| Op JaLeA JusAnod saz[o,nb sreur ‘seul 39 
JaLIAgy ‘Jorauel inod saurgui say JOS ssuOodal ap soyTIqeqoid 
sa] aonb juauWeTe39e astogid sfepoul a_ “jJjag/Axo1g sodnoi3 
anenb sap anne | & UN] op JUoWIWepUSdapUT JOLIeA Op SIeUL 
@ JOLIAQJ Sp 39 JOLIAQJ & JorAuel op jUsWIasUeYD op SaITTIqeq 
-o1d xne youLied p a[apoul a] ‘sgJapisuod suoAe snou onb 
SO[QRIJIUSPI SoTQpoUL sap yeI9UaS snqd 9] ISO p B[QPOU! JT 
‘aryne | @ SIOW UN.p 39 
anne | @ odnoiz un,p saye3e ang sinofno} jUdATOp ssuod91 
op sauiqeqoid soy ‘slojoinoy, ‘stow a] UOTas Jo odnol3 
d] UOTIS JOUeA op JUoUESURYO ap saz[Iqeqoid xne duUOp 
jouttod afapout 09 -““!?u x *!“x anb alos op juouraSueyo 
ap sauiqeqoid say inod g}1euUONe}s op onb syIgugsOWOY ,p 
JUL} SoJUTeIUOD Soy WNpel ¢ s[epow aq] ‘d so] sno} inod 


(ST) Fay ela gy Uae ane ily 


B oseUeARp (Z[) aIUTeN 
-U0D PI SUOSINPsl snou ‘7 s[epour sg] INOg ‘d ‘AXOId/JI9S 
odnois BJ] UOTAS IOLVA OP SICUI B IOLA] Op 39 JOA 
B JotAuel op snudAins sjuauesueys xne youtlod nb 90 


(v1) port sd inod 7y - “l¥y 


B (ZI) aUleNUOD eI SUOSINPEI SNOU | I[epow 9] INOg 
‘d sat sno Inod 


(€1) ¥(2), . FA%, _ FG4 _ ale, 


jo 


(Z1) Te rey chal GS Ee 


SOJUTLIIUOD Sa] JURSOdUII Ud (P) ap JUTIQO UOJ onb 


(11) hy Hy By dy PZ 9°9°0'dy 
ee ig 
jINS SUTWOD 
d1199,$ jnad ajapoul aD “(stow ston sayz inod sanbryuept 
juos so[[o,nb alIp-e-}sa,9) soleuuoneys 39 (g odnois 9] 
uoTas sed jUdIQIJIP OU Sata nb aiIp-e-1s9 9) sougsouOY UOS 
asuodal ap say{iqeqoid say 1a JUaUIaSURYD ap Sar[Iqeqoid 
sat aonb asrogid jt ‘aydunts snd a] WOTW eepoul 2] 3S9 
9SEq OP 9[QPOUI J] ‘sagUUOP ap SaTquIasUA SION sap UNDeYD 
Inod saxaduros snjd ua sntd ap WOT Se[2poul ap atigs 
oun ,p Juowa)snle | ap syei[Nsai So] onbipur | nesjqei a] 
“sogigpuod UOU SUOT}BUIT}SS SOU SURP JUDINSIJ JUSUL 
-odnoiz3 ap sjajjo sap stnas anb sripue} ‘ayesaut uonelgpuod 
Re] e.nb uojjnueyoe,| op juowodnois ne jue}, s[qenquye 
SOURLIVA BL] OP UOTEIUDWIZNe | dIQ]JOI SqD ke] Op osepuos 
op urd np iayja,, onbsind juapnid 3sa} un ayuasgidas 
v[I9 anb SUOAOIO SNON “IIUDA B asATeUR | SUP sgtIdoidde 
DUILUOD IPIOPISUOD BIOS IIBIICIS DIIWIOP 99D “suOTeUINSS 
XNO9P dIJUS DOUSIOIJIP LP] JOIe[OEp op JUSWIOUT Ne %S ep anb 
1oin{d %¢ op UONLOTJIUsIS op NRdAIU UN JasI[yN ev dISISUOD 


€9l 


ajugeatnbg a139}e.ys ou) “UaWednols np a}dwW0d Iu} Inod 
sadA}- sIndaLd sap aURSTJJNS UOT[JUI OUN JOUTe.NUS eIAOP 
w(¢‘1) red WA sedAj-sinowe soy Jot[dnyynus op wes J ‘¢‘T 
sed juossedap ou Sgd B] op sande uoe[ndod x] ap suo 
-PUINS9 Sa] Inod asepuos ap uryd ap sjajyo soy onb gnbiput e 
(6-1 ‘O00Z) stUf-sIeIq Sep snsusD sy} Jo neaing 3T “SdD 
e] op satjoe uoNELindod ey] ap suoneumsa sop snred B g[no[eo 
asepuos op urd ap jajjo un sed JA) soouewea soy sJo1d 
-1y[]Nur suoAnod snou ‘auawousud 39 op juoWeAneWTXxOIdde 
aidur0 319} Nog ‘SqD PI op uOT[HueYDE | suep juowIodnoI3 
OP 19JJO,| Op Douessieuuod op onbueul Jed saguinso-snos 
ang ,p juonbst odAj-IneL9,| ap VA) SuOTeUINse soy ‘ayduuts 
allojyege aseuuo][NUeYdE ,p sasauJodAy ,.p UONDUOJ Ud sag] 
-nojed jos satja ,nbsing *‘sadAj-sineL9 sop UOeUINSa | 1S9 
deigpuod uou asdjeue oun & sInodeJ Np yoodse ane UA 
"2]210} 
uonendod ey] & Sg BI op UOT[NHULYSE | Op R[op-ne soUdIOJ 
-ut oun Inod sgtidoidde ouop juos sgsgpuod uou se[n[99 ap 
S]USUIBIQUIOUEP Sap Inod snossap-Id sajuasgid syeI[NSEI SIT 
“Q[RULLIOSP SUIZISION eT B Nb ajsajlueUl JULID ,U DOUDIOJJIP P| 
‘sonbnuspr juowonbiyeld yuoreig suonsodoid op su0j snos 
SOQUILIGXS UONIIJISSe[D Op INdLIO,p suONeUINse soz onb 
QAIISGO SUOAR SNOU J2 SagIgpUod UOU 39 sagIgpUOd sIOj PI & 
SoQuUOp Sap Inod UONROIJISSeID OP SINILIO SO] SUITS SUOAR 
snou ‘agigpuod uou asdyeue oun Ins sogpuoj o[e}0)} UOT 
-ejndod ey] inod saouasgjul .p SyIpreA P| JOUNWIeXd .p UY 
"syeq[Nsol 
So] SUBP SBDUdIQYOOUI sassnej op siINpold op jesJonbsts 
SOIQILID sane soa_ Inod sagigpuod saguuop ep uONes] 
-11N | ‘sagigpuod uou saguuop sap a3Ixe asAyeur | Sp SUIOU 
ne oed oun ‘juanbgsuod seg ‘afqiuodsip jsa,u sprod uno 
-ne Ja sagigpuod sed yUOs ou f 39 € SOIQILIO SO] JoNTeA BULA 
-198 SD P| OP MIIAIOIUIOI 9p SaQuUOPp Soy ‘JUDWIDWIgIxNEG 
‘sogigpuod uou saguUOp sap INs sagpuo} sayno} ‘SqD 
Rl Op UOTPIIJISSe[O Op INI] ap JUSUOINILIQIUe S99zI9 
SopNi9 Sap SdgIN SUOTRUITISA xNe WO TIA SUONPUII}SO So IOI 
-eduioo suoanod snou “juswWoIgWsig ‘s}uevOduII saseUeAR 
XNOP IJJO VOIW eeIens ev] Inod seggpuod uou seguuop 
SOP B SINOI JJ ‘SIOJAINOT, “(SRG] UOseIT_ 39 3B0[D) siniq 
Sa[N]Joo op xnej0} op anb 30jnId a][eyoe,| & SIWI9I 39 SoIOp 
-uod sa[n][9o op sJUsWIaIqUIOUEp ap apre | & SGD PB] Op sajes 
-9UI soi[iqeqoid eB adeuUOT[HURYDE _p sINjOMU}s Be] op a}dWI0d 
QIPUdI Op II qQISsod js9 |] ‘JUSUIA}OeXA YSITSpOU ag sed ynod 
dU SD Be] op oxa,dwio9 a8epuos ap uryd 3] anb ayos ap ‘ayd 
-UIIS aIOJegTe SseUUOT[HUeYDE UN yUuasoddns WOT e[epour 
op juowsysnle .p satoisoid sane .p 19 JA) [S10130| VT 
‘JUDWISANIEdSaI ‘SIV 39 JOWAYJ “JatAuel Inod Ande | 
OP SIA-B-SIA UONENIIS BP] IP UONROIJISseIO el JUoJUESgIdOI 
2 € ‘V 00 ‘DxXYXVXd EP gSIOI9 1} 9] Jed TULJop se[NI[ed 
op JUOWIsIQUIOUSP B EXEXEXxp Nes[qe} uN jIR}9 JaNUS.p 
saguuop op s[quiasud,| ‘g “UoWadnoIs op s[qeLIeA o[Nes 
oun juryOdwi0s WOW 22poul un .p juowiajsnle | Inog 


VOI S2[Qpowi sap yusutajsnfy p"¢ 


*(uenb un. p nal] 
ne sJot uN) gaa[9 snjd yusUIEIgg] Iso AxO1g odnols a] suep 


0002 eiqwecap ‘aj}enbua,p senbiuyoe | 


UOT[HULYDY ,[ IP SSIQUIDUL SP SIQUIOU J] ‘966] 19 S66I INOg 
"AXOIg AMSOP] 19 JJES APSO] Sodnosd soy anus yuousle3a 
sgid ned e styredal jue} UOT[HUeYOY,.| ap SaiquIoW sone 
say ‘Axo1g odnois 9] suep uomAus uenb un ‘JJag adnois 9] 
SUEBP 9ANOI 9S UOT[MULYDY,| 9p UOITAUD SION} UN ‘E66 INog 


souuosied [CL Ip 9661 
souuosiod /p¢ 6h °S66I 
souuosiod [67 Sh °€66I 


:OJURAINS LT ISO BSATBUL OIJOU SUP Sasi[n ssguUOp 

OP Sa[qUIOSUS SION) sot Inod suOT[HURYSE sap aT[Ie1 eT] 
*(SNssap-19 p 9199) suOTBUIT]SO 
Sop SI[IqQisneld op 919}119 np aIped 9] suep asayjodAy o1199 
SUOIJLIQA SHON ‘(p=d) AxoIg odnois 9] aonb uoneorytsselo 
OP SINILIO .P SUIOW dIOdWIOD ([ =) JJaS adno1d aj anb 90 e 
suOpusye snouU snou ‘([ 66] YsteyeoyoIN .C a[dutoxe red) 
SOINSLIQIUL SOYdIOYIEI op nus} aJdwi0D *7 uoNdes FY B asT]NN 
suoAe snou onb ‘5 ap nol] ne “Yuowadnoss ap s][qeiea 
B] Jayuesgider nod g jueUajUTeW! suOsTN snou enb Ja}0U Y 


(AXONUd) uoneinsoid sed 
SOQUOUL $9]NO} JUOS SMOTAIOIUI SION SITIS p 
(AXOUd ATLSOW) vonesnsoid sed 
SOQUOUI JUOS SMOIAIOJU! SION SOP xnapIs ¢ 
(ATAS ALLSOW) uonesnsoid sues 
SOQUOUL JUOS SMOTAIOIUI SION SOP XNIpP Is Z 
(4 TAS) uoneinsoid sues 
SOQUOUL $9JNO} JUOS SMITAIOIUI SION SOTIS | 


"INS OUIWIOD JUIJEp as ‘g 99]0U 
‘Q[QeUea 97395 ‘uonendod ey] ap sytouasola}9y | op a}dwWi09 
nud} xno a @ nb SgD ke] ep UoNeINd0I1d sues NO ddAe 
OsUOd9I Op INIJVIIPUL | SP I9J1I} B[QULIVA kB] 3S9,9 ‘1d-Sa][IO 
Teg “90e1 BP] 39 (UONeINOOId sues NO daAv) asUOdaI ap 
adA} 9] ‘MAIAIOIUI PP 9POU J] ‘aILIV[ODS eT ‘axas a] :studuIOS 
A ‘so[Qpoul so] suep aiNpoUr Be UOTIROIJNeNS op so[qeleA 
OP 30 SIIQeLIBAOD Op SIQUIOU UTe}IDD UN QUTLUeXd SUOAL 
SNOU ‘9}I9USZOIP}9Y 91199 op a)dUI0D sIpUaI Op UIfY “sougs 
-ouloy sed yuos au JUsUIaSULYD ap ja asuodal ap sayt[Iqeqoid 
sot onb gnbipur e (266 ueseuUe]y 19 Aloysng ‘IouWloIg) 
aureurutfgid askyeue aou ‘slojainoy, “(aiugUuasOUOY | 
ajedde uo nb 99 389,95) afqro uonendod ey ap sauuossed soy 
saqno} inod sanbnuept yuos “a ‘*!“x “suauraSueyo ap seit] 
-1qeqoad say onb rsure ‘7!’ yo ‘“'7x **!"xz asuodgu ap serttiq 
-eqold sa] anb astogid ayduits snjd 3] YOTW 212powl oT 
‘askTeUR | OP SN[OXS 939 1UO SIOU SIO) 
Sep Sinatsnjd no un Inod Janus yno} eseusUI Np JUoWIesURYS 
OP Sed sz] 19 BsuOdaI-UOU ap Sed Sa] ‘ssgUUOpP ap Saguue 
SION Say Inod (sIeUr 39 JOLIAQJ “JatAuel) syundgsuod slow 
SION) Ins askyeue | Joyod otej op ISIOYD SUOAR SNoU ‘9IIOW 
op osAjeur | Inod uoT[HUeYo9 | op sI[Ie1 eB] WesINpaI SoguUOp 
op stow onenb e sinodal a] onbsing ‘SdD eX] ap sjiino 
-9SU09 stou anenb no ston ap SINOD Ne SagMIIAIOJUI SOUUOS 
-Jod saj soyno} yueUsIduIOD saguUOp op So[quiesua Pp xIOYd 
UN SUOIAR SNON ‘a[Qpou! ap soNQuIeIEd sop 9IT[IqeLJNUSPI | 
Inod S9aNNdgSUOD SMOIAIOIU! SION] SUIOU NV 93IXe VOTN 
RB] ‘sJynogsuod siour ayenb ap oligs surgIxnep auN Pp siNod 


NV SQMOIAIOJUI NEIANOU 9p jUOs sind ‘stow! yNY JUepuad 
aygnbua,| ap omsed sed yuoj ou ‘syynogsuos stow anenb 
OP SINOD NE S9MOIAIOJUL UOS SqD kB] ap SsaseugU so] 
‘sduis} op aporied ay190 1nod sayqis 
S999 JUSWO]LOVJ JUSTLID Sg ke] Op saydo1dde1 uOU MATA 
~JajUI91 ap SoguUOp sep onbsind ‘9g6T 19 C661 ‘C661 ‘SdO 
e] ap soguue sion Inod WOT eS9IeNs eT SIJLI9A SUOAR 
SNON ‘JUSUIaTUBUTSI 9] Sgide jo JURAL SgD PI OP Inada] 
g1edui0d suOA® snou ‘aysIaYyse1 oJUasgid eT op ediould 
Jnoafqo un.p sed ossi8e.s ou pi,nb uatg ‘¢66] eB JOU 
-SATJeIVAUIOS 766] SQIde SqD | op UOHBOIISseIO op INdLIO | 
ap (UOHONpel uN yUoUEsID9I1d snjd) soUSIETJIP ouN JaAJasqoO 
B sUOpUde SNOU snouU ‘suOTeIOTTAWIe sed ap sduanbgsuo0d 
SUD “$d P| Op JUSWaTURUAL 9] ILIDEp B (P661) e8II0Y 
“QUATIO’ | OP SIA-V-SIA UOTJENIS P| JOUTLUAI9p & JURAIAS SUOT] 
-sonb sop UoNeINjONSaI JBAv ‘a[QUIOD US PUOJ ap sgruLUISI 
919 & Sd<D Be ‘(Anoyeurpso Jed a9)stsse ooeyd Ins MdTAIOIUT) 
OVdI.I OP SAND Ud STU eI Op oIped 9] sULP ‘P66 UT 


SdO &] 9p saguuop ap sajquissuy = ¢"¢ 


“O[Qpouwl np 
UON]DITIS Op snssad0I1d Np sjyeI[NsaI ST 19 S_D LI op seguuop 
OP Sa[quUIasUd sa] sILIDp Jed suoSuaWIUIOD sNON “sIprTeA 
OP SdIQIIO S99 ap 3}XdJUOD oI sueP WOT voNestfepour 
9P S}eI[NSQI SOU ap SUOINOSIP snou ‘aJURAINS UOTOVS eI] Y 


“IPULIIJUOD 3Sd aJUaIedde dyprfeA Ing] ‘sayqrsneyd 
JUZTQUIIS SogSI[PPOU sUOTeUITsS soy onb juene 
Inog ‘ai[Iqisne|d ve] sp UoNOUOJ Ud saNTeAg 9 SOUT 
-Wx9 ang judAnod uOTedIJISse[O ap Inda,p s[ijoid 
sone q ‘uoneinooid sues sasuodgi sap onb sajoexa 
snjd juatos datjoe uoTe[ndod ey] ins suonsonb sap eB 
uoneinooid red sasuodgai sap anb ayqisneyd nad afquias 
[I ‘ojduroxe Jeg “SUpl[eA k[ Op sAnaId9_,_p IAIOS JUSS] 
-e89 jnod asuodal ap saiyiqeqoid sap suoneunss 
sap (ajuoredde oyIpryea ey no) ayiqisneyd ey ‘uUuq 
‘UOIJLIISSE[D OP ANI119,.p splyoid sap aziqisne[g ‘¢ 

‘g[qeyea sed jso,u SULOU 
ne saizajens sep oun nb onbipur piosoe syqrey un nb 
SIPUL) ‘SOPOYIOU XNoP Sop SIIPI[eA P| SULITYUOD YWOTN 
adA} ap 19 MAIAIOJUIOI OP SUOTILUIT}SA SOT 9JUS p10d9e 
uog uA, ‘sagyooiddel uoU SMOIAIOIUIOI op SogUUOP 
ap sanuaigo ‘SgD eI oP MOIAIOUIOI op ouWeISOId 
np sagin syiqeiyy e] ep suonewnss xne USUI 
-Q}D9IIP SUTIGeYJ V] op SUOTLUIT}SO Sad SUOTedUIOD SNON 
(6) @ (L) OP Ople.| B 9OUSIDYOOU! Pp sdIpul,| ep sUOH 
-RUIN]S9 Sap I1u9}qO Inod UOeOTJISse[D Op INeLI9 | ap 
WO'IW suoneuinse saz 19 In[d suosT[MN snON “"VWOTW 
SUOTIRUINSS Sop supIeA ek] op uoNenyeag,, snod 
Jaye -INH] 9p UONPUI}SE Pp SPOUISU L[ OP SUp![RA LTR 
jadde sed yey ou oshTeue a9199 ‘stojoinoy, “seyooiddes 
UOU MOIAIOJUIQI Op sogUUOp sop INS segpuoj 
suoneuInss sop & WOTW XI 9p seit) sucess 
sap aieduiod |i Jeo a1aqWIO sUIgISION ne s]qQe[quies 
1SO DIDI 9D *9IUIAIYOIUIL,P IIIPUI | IP }$9}9.1-}S9} 
}2 Sagslapour suoeUIsd say o1jU9 p10dxIY ‘fp 


ajualeE] sinjonjs ep euua!AoyeWw asAjeue,| ap syipeA :AIeysng jo Jawelg ZOL 


“QUUDIAOYIeUL asaujodAy,| & saTjamusag suoN 
-OBIJUI ,P SIA-B-SIA SO}SNQOI JUSWIIATIRIAI JUOS ssuodgI 
ap saiiqeqoid sap WOT suoneunse soy onb sours 
-NSSB UIeLIIO DUN JUSUATeS9 WUINOJ M\-H 19 VO'IN 
SUOIJCUUIISA SOT US BOULPIONUOD dO} UL) ‘BTQLTeA 
sed 1sa,U SUIOU! Ne saIsa}es sop oun nb anbiput pro 
-o a[qIvj UN, Nb spur} ‘SopoyaW xNep Sop IIPI[VA PT 
SULITJUOD SUOTJLUUITISS Pp SA[QUIdSUd XNap Sd] 91JUS PJOD 
-08 U0d FY] ‘VOIW epoulew P| Inod sagsi[yN safjeo op 
sojuepusdapul jUuOS AA-H epoy}eu ve] Inod saguuop op 
Sd9IIUD Sa] ‘ayoIoYydeI aJUasgid ey] suep ‘sn{d aq “a[ep 
-OUL Np aI]IqeipHuapr | Inod suustaoysew ssoujodAy ,p 
sed o81xa,U M\-H epoujeul ey ‘ojdurexe seg ‘soquol 
-9JJIP Sai] JUOS UONRSITapoul ap sasauiodAy soy ‘o}U9} 
-P] aINjONs op sejagpour sap e joedde xnap sajno} JwUOoJ 
M-H 32 YOTW Sepoylgul soy Is oUIgW “(8661 *9661 
YUIMISeH 32 IepouIS) sayooiddes uOU MOTIAIO\UIOI 
op saguuop sap e jadde yrej Sg_ ve] ep esuodgl op soz] 
-Iqeqoid sap uoreUINse Pp (O86T Jo1T2AA 39 INH) (M\-H) 
Jay@AA-IN}] ap spoyjeul vy ‘asuoda. ap sazyiqeqoid 
Sa] Anod 19}[VAA-INF{ IP }S9}91-}S9} suUOIVeUIIZSa 
$3] 32 WOTW suoneujsa say o1ju9 p10ddy 


‘9661 3° S66I 
Inod Ina | sILIOEp JUSWIETeB9 IWeIAOP €66] Nod inal 


-19 | JUDWIa[qeUdAUOD jLIDEp INnb sest[apow sinjonI)s 
oUN “UOWdIUUIAI NP Jajjo,[ snos asueyo sed yuo .u 
SdO ®] ep esuodal op ineLia,p seoinos sayediouid 
SOJ IS ‘SlOJoInoy, ‘€66] Seide Jasueyo nd e ssuodg1 
ap sINdae sap Inajduie | anb ayOs ap ‘(inayeurpso sed 
daisisse sovyd INS MIIAIOIUL) OVI, oP UONDUO] us IU 
-PUISI 919 & Sd &] ap auluidun aireuuonsonb 2] ‘¢66 
saidy ‘uoneordde ajuesgid ey inod gynoyjIp oun 
BOdWI0d spoyjJeU 939d “oINj}ONIS VS Op SIIPIRA P| 
JOWNIJUOD B VdULpUd} JeINe oN | B BQUU OUN.P aap 
* -OUI NP p10998 TJ “a[Qpoul np ajuepusdepul UONPdIJLIDA 
ap onbruyse} 93309 ap yoodse un 4sa JUdWIgIedgs sQuUe 
onbeyo ap saguuop xne s[gpour swigu np juswa}snf 
-® | ‘apne ajuasaid ve] sue ‘(eu09y} ve] Jed onb 193n]d) 
saguuop Ssgj Jed sepin3 s[apoul ap UOTJOeTas oUN Jo Bap 
-OUl Np sseNoureIedins un JoyIAI,p youed spoyjou 
NID “(OEE ‘861 JOIN 30 Joddny ‘winequiepy aid 
-wioxe Jed JI0A) UONest[apour eT Inod sagstjyn ssguUOp 
sop sojuepuadgpul juos Inb seguuop sep Anod s[epoul 
np juowsysnle | Jan[ead & djSISUOD a[epoul UN .p UOT} 
-epl[ea op onpuedal apoyleul dU) ‘SqD PB] Ap 919Nne | 
g agUUe UN p IJQpoul np yusWajsnfe | ap 9IPYeA 


‘IN] 99AB JUSPIOIUOD saguuOp szz aT[enbey] 
SULP SINSOUL LI 39 19-IN[9O ap gpuojJ-UdIq J] JoN[eAg,p 
ulje d[Qpoul np uowa}snfe | ap sonbysouseIp soinsoul 
sone .p onb sure yuowajsnle | ap aypIyeA e] op gues 
-IYD [OUUONIPeN s1Q}9 a] sUOSITN sNON ‘ayenbepe 
uosej op soguuop soz aisnfe [I,nb 39 (9iTeas eB] DOA 
JUSPIONUOD 9 SO[qeuUOSIe JUSIOS SasaujodAy say aonb) 
g[qisneyd qos 10-In[a0 anb 3sa afapoul UN ,p dIIPITeA PL] 
OP IIILSSID9U UONIPUOD UL) ‘s9SI[apour sdIsoUsLIG, 


LOL 


:$019}110 buts op apre | & 
SdD BI AP UONROIJISse[D op INdd,| OP WON Suoneunss 
Sop SUpPeA ek] SUOUTWIeXS snoU ‘a}x9} NP o}INs eT suUeG 
“VOIW SuoeUIT}ss sap steIq 3] JOWINSE .p 
gjqissod sed jso.u [It “uanbgsuos Jeg “(O86 JOUTOIYIS 
onb Isure ‘766 ] UBUISIO,J 19 JOWOTg “8661 ‘9661 YLIMIseH 
yo mepourg afduroxeo sed MOA) SgD e] IP opmnoexO,] 
op uoTyeneag | Inod g}da00e yUOWaTeI9UES UOTeI9 UNONe 
a1STX9,U [I JUOWOINELIQ\Ue SopUUOTUSUI sUOSTeI Sop Inod 
‘\uouesnoInsye| ‘SdD eB] suep sanoy uO] aonb sade | 
OP SIA-B-SIA UONRNIS LV] BP SUONROIJISse[O sop INL .p 
XNBOAIU SI[UILIGA Sof JAIJol ‘,, a ‘INoWa.p soifiqeq 
-O1d sap Sa9sI[apOUl SUOTILUINSA So] IS JoUTULID}Ep suOIgdso 
snou ‘uougstoaid sng “SdD PI op UONPOIJISSeID Op INDIA. 
ap epnj9,| Inod aN 19 SANRULIOJUI SO YOTW oisarens 
RI IS JOUTWLID}9p SUOJdWIOD SNON “(R86 SIOAZJAI) SSTTNN 
UONCUINSA ,P SIISICIIS SOP SPI[eA PV] SpsIOge JUSUIS}Q,dWI09 
sed qU0 .U $gD PI OP UONRIJISse[O Op INA ,.| 9p SINSOU P| 
ap sojuapgogid sapm9 SY] “WOT e89IeNIs P| Op SUPITeA eT 
Jonjeag ,p Juoalqo jedtourd sumu09 & gsodxa yuasgud aT] 
VO'TW 2poyjgul &| op gpIpyeA &[ op UOKHeN[eAY 7¢ 
(OI) ‘yu - peel 
QINUOPT,| Op Ople.| & fF OP y SoLosgivo xne 
sgroosse ‘"g sg\ou ‘aInsaul ap steiq sap y 1N9}99A 9] JOUINSE 
suoanod snou ‘aguuop yt 9p UONeWINso oun Inod ‘uyuq 
"saLlosayeo 
Sop NRSAIU Ne SddIpUI Sep daJgpuod suUsAOUI BUN 39 (096 
udyod) ueyod ap eddey syr[iqeiy ve] op oINsoU PT ISO X NO 
(Q00z AJousng 19 Jazulg ‘ssaH) X — | B BeBe suonsanb sop 
NVIAIU Ne Y}I[IQeIJ-UOU P| Op DINSOU OUN Jsd [eQO|S SOIpUI TJ 
“OFT ‘Q0UdIQYOOUL,p [eqo[s sorpul,| sejadde ‘saguiqui0s 
Y Sa089w9 say $ojno} Inod souss9YOoU! | Op s[eqoys ons 
-oul oun nb surgut ap sodA}j-sinaLa sap So[NWLIOJ So] TWINOJ 
® (16-88 ‘S861) SIuf-sIeIq Sap snsusDd ey} Jo neeIng FT 
‘juowiaanjoodsar ‘” "u op 19 “ut op aqusre] 
ainjonns op suonewnNse sap yuouSissp ” "4 19 "u no 10 
jD#D 
(6) Fey. ROT dps 
Jed tuljap Inq DdUdIEJJIP op xne} a] aJUesgidol ups no 


("u - 1)"uz 
(g) aS ill Sled 
dps 
yso "7 ap INdJeUINNSS UA ‘YD = FP BIOZ9\VO 
BI Inod sousIgYOoUT ,p soIpul.| ‘"7 wos “(Sg6t sNsUueD 
oy) Jo neoing ‘S'—Q) IIOA) SqD kX] ep aatoe uoNendod 
RB] Ins saguUOp sop dI[IGeIJ LP] Op S[[OUUONIPeN sINsoU e] 
ajuesaidal inb ‘y - | = ‘ddUdIQYOOU!,p SdIpUI | SUO[NITeD 
snou ‘asAyeue aNj0U INO * ,,,2f Op UOTVUIISS 91129 B (E861 
Ipasulyog sduroxe sed MOA) soyjeniiqey uoneUNss.p 
sopoyleul so] yuenbridde ua ayANoe | 3p SIA-B-SIA UOTeNIIS 
9)n0} nod ‘y ‘ysa}aJ-3S9) SNTIQeIy Vl JoWTysS suoANod snoN 
‘(L) ep 12 (9) Op opre |e ,,%# Op UONeUINSd | queyouied ‘yt 
OP 12 y,,a OP VOUR[QUIASIVIA Op WINUITXeU Np SUOTVUITISA 
Sop PIIWINOJS Sd L] Ip sayeulpnysuo] saguuop sop WOTIW 
RI ‘aquapaogid uoNses ey] e snbijdxea suoae | snou sWIWIOD 


0002 eiquadgp ‘ayenbue,p senbiuyoe 1 


‘yi “UOTIVOIJISSe[O ap 
sai[iqeqoid sa[qeiLI9A Sap INd99A Np ja ‘yi pri ‘Sotfouuon 
-Ipuoo ssuodgi ap saifiqeqoid sop sole ev] op unpold 
ne g[es9 3s9 * pylt “SO9ATOSGO SMOIAIOJUIQI-SMOIAIOJUL SOP 
UONCIIJISSe[D ap Nvs[qQuI Np syI[Iqeqold ey onb aip-e-1sa.9 


(L) exe pvt 


SIO[S SUOAR SNON “UOMPOIISsEyO 
@P sguiqeqoid sa[qeilIga sap YY INB99A 9 ‘1 10S 
u ‘Sd][2UUOTJIPUOD saIT{Iq 


‘XIV 
-eqoid ap inojo0a un .p ogsodsuey el ousIsap /(,,,2) NO 
(9) Gar) xiv! = x yet 

SUOAR 


snou “(E86 IpeIsuIyoY) safay[esed sainsowu sap aseyjod 
-Ay,| UoneUsWINDOp eI suep sgjedde *!’u = (x= V|,D= |" 
‘p= ¥)1g= ,”'u anb suosoddns snou 1g “(x= X|,0=,V 
‘D=PV)ig ‘a[nyjeo ap saiiqeqoid sop y x y someu 
Bly ppl WOS LY X LY SOQAIOSGO SMOTAIOIUIOI-SMOTAIOIUI 
SOP 9SIOIO LN] ap nes[qei 9] ‘“, KF WOS “(= ,P)Id='U 10 
(o= pP)id= "u SUOSSIUTJap Snou {UaWaANOadSal ‘MOIAIOIUIOI 
eR] Inod jo euIsi0,| nod 9IIANO | AP SIA-k-SIA UOTIENITS 
R] Op UONOIJISSEIO PI ‘, f 19 F IOS ‘ayUapegogid UOTIeS BT B 
OIL1D9P UONLIOU FT SUOST[NN SNON “MOIAIOJUIDI-MOIAIOIUL ,P 
SoguUOP Sop asAyeUR | Op Seelt} SUOTRUITISS xn UONLOIJIS 
-Se[d op INeLd,| ep WOT SuUCHeUINse sap uostereduros oun 
SO YOIW aisaiens ev] op uonenyeag onou op omued sug 


UOnLION T'¢ 


Sd)0 VT VY NOILVOITddV ‘¢ 


“SOPOYISUI SIO Op SIAN Ud ISIUI PT DINSse ‘dJUBAINS UOT]OS 
RI & SD BI Op saguuop ap sa;quiesue xne snbijdde ‘was 
[A1D1ZO] VT “SU [IQeuINse Ina] ap SUOTIIPUOD Sa] LDP UO STI 
10 ‘JTQpoul 99 ap sanowiesed soy JUSS .p UlYe J-q OWL 
-o3ye | Jonbrdde .p yueyoutied ajnuLioj eT TWINOJ JUO (986) 
MNT Op 19 [Od op ueA ‘DdULTQUIASIVIA Op WINWIIXeW np 
UONLUWINSS Pp SOpoyIeU op opre | & SPUINSd ag JUSANed slap 
- OW op soneweied soy ‘WUdtIOINALIQ]Ue SagoUOUD sosauod 
-AY Sop NYA UY ‘2 19 g ‘Dv ‘8 SadIpUl soz Inod swe) sop 
wnpoid a] ousIsap [] 19 sperMoUT[NU 9}UR}SUOD BI ISO ¥ NO 


2‘q‘n ‘3 
(S) eet I] ¥=(OgKD)id 


ISO DYKD nes[qe} 9] Inod souv[quiestesA op uON) 

UO} P| ‘[eMUOUN [NU sseUUOT[HULYSE UN Pp sped | sue 
"JUDUID}OOIIP 
UOJ 9s JUDWIAdNOIS ap sjqeueA oUN,p snjd B SUOISUd}X9 SOT 


By z 3, ‘3, 
(as a1?) a3 | Zp FANG F814 FBIM BX yy a mee 2‘q‘p at 


"INS DUIWIOD DYKH nee[qei np (2 ‘q ‘v ‘B) a[NIJa0 
BI SUP BSSPID OS SqD PI Sp UOT[HULYOE | op eAquow uN Nb 


gufiqeqoid ey] JajnuUL0J suoanod snou ‘sasayjodAy s90 ap 
nus} aJdwiod -asd[eur | op sinod ne JUaWAINELI9I[N IWUTEp 
% anbuosjenb (uoneoynens no) yuoutednoi3 ap s[qeueA 
oun “4 Og ‘WsWaANOodsal ‘sreU Jd IOLA] ‘JotAuel Inod 
(SO99AIBSQO UOU) SUOTBIIJISSe[D SaTQuILIGA So] ‘7 19 { ‘X10 
*S99AIISGO SUOIIOIJISSLID SO] ‘9D 19 g— ‘V IOS ‘asATeuR oNOU 
OP SoJURI[NSOI SITqULILA So] JUOS QUUP | op aNSOUIL JorUeId 
Np stow anbeyod ap sinoo nev syANOL | Op STA-B-SIA UOTENIIS 
eB] ep isd Inb 99 nod SgD eI ap suoNvorisse[o sa] 
‘g9sodxo juasaid 
J] SULP SUIPI[eA ke] OSejURARp IIpuoCjoidde ua.p JajUa) sues 
(¢) Jasoddns juowleyes9 suoyye snou uanbgsuos Jeg ‘ayqrey 
ZISSE SIOJOINO} JURID JOJJO,| ‘UONeNIIs op sed yuasuLYo ou 
Inb souuossed sap o][99 & OINSLOJUI SIT[IQeIy oUN yUD\sayTURUT 
PUAIIOV | IP SIA-B-SIA UOTJeNIISs op JUssueYo Inb souuosiod 
Sa] onb suONRoIpUl soUTeyII 9ANOI UO SII “aIANOV | ap 
SIA-8-SIA UOTJENIIS Sp syUsWUasULYD ap sedi} SIOAIP ap BRIO] 
ouepusdspul | JOIJLIQA op UlJe ajUDIL] DIMJONNS op soap 
-OU Op opie | & MOIAIO\UIQI dA} ap seguuUOp sap je Joued 
uN .p USAOU Ne Sal|[londe1 sagUUOP sap deUIqUIOD UOTING 
-INSIP B] QSI[QPOU UO (L661) BUTeYyasueT 19 [Od op uURA 
‘9[Qe[QUIOS UOISN{DUOD UN 91 yUO ‘aATOe UONR[Ndod ze] ap 
SOLIVUDDS OP SIQUIOU UIeLIIO UN .p UOTOUOJ US aseuOdAY J 
Op Vssajsnqgol ke] sIpny9 yuo Inb ‘(¢gG]) OF Y 39 YBUIS ‘a[quu 
-uosIel UoeUTXxoIdde oun Jajuasgides a[quias asguiodAy J 
aonb njouos ev slaXkayq (S861) JOUTIeZ 39 pMogy Jed sestyuN 
UONLUISS Pp 139IeNNs Pl Op OpNie uOs suLp (YR6]) SISA 
Jed $gD 2] & JUDWISATILIAI SOUTUIeX9 919 B ‘aTeOO] soUepusd 
-9pul | op assamodAy,,| sgjadde ‘asayjodAy a9 “stow ston 
Sop one | & UN] op aJULPUddapUT Iso SANOL ,[ OP STA-R-SIA 
IdAJOSGO UONLNIS LI IP UONPOIJISSe[D Op IndLIa,] anb no 


(¢) 232 B95 S195, 


suosoddns snou ‘uljuq 

‘LO[GWId SUBS DUTUIOD JUDUII}DIIIOD B9ssRID OS 3 ednois 
3] suep ouuossad oun,nb gyiqeqoid ey aso &* 3!2= "x 
‘QuUIQUI oq “(Z = _Y) lo[dwia sues ayITeOI UA JsO O[[9,nbsIO] 
(1 = P) egAo] dua our09 S$ gd eI Jed agsseyo wos 3 odnois 
9] suep esd gun,nb ayyiqeqoid ey yso ©7* 71TH Py 
asuny °7 2!°ar yo “2! 4x inod Searigaes SUONIULJEp Sop ddAv 
‘(x= Ss =D (0 =P)id= "wu red suoneoijisseja sao 
ap sunoeyo Inod asuodgi ap saitiqeqoid so] suousIs9p snou 
‘snssap-9 o1[qei9 UOTe}OU BI BP UOIsUa}Xe OUN,P Opre.| Y 

‘yuauiaatoedsai ‘¢ 19 Z sapotied say Inod ' ky yo ‘"g osuo 
-91 OP SINJBdIPUI Sa] Inod sa[qe[quIes SUOTIIULJap Sop 9eAe 


VA, @& Ge tne 
x3 |9'q ‘Oy 


| aporied e] op sinod ne sANoeUI 
QUIUIOD B9SSEIO 1S9 (7 ‘3) QUUOSIad BT IS ¢ 
| apoted ve] ap sinoo nev iojdwesues | oz 
QUIUIOD B9SseID IS9 (1 ‘3) QUUOSIOd BT IS Z 
| apotied ej ap sino ne ago] dure 
QUILIOD dgSsEIO ISA (1 ‘S) QUUOSIOd RT IS | 


no uawisatoed 

‘ 12 (18 «! 13 
-sal ‘¢ 19 Z ‘| sapouad say inod “19 ‘“ g *“ p sagi0u SdD 
BI & JUSWIAATL[AI SANE] OP SIA-R-SIA UOTENIIS BP] OP SAGA 


-I9SQO SUONLOIJISSE[D SO] JUBUDIUTLU SUOCIZPISUOD SNON 


a]Ua}e| auNjonys ep euUaIAOMeEW asAjeue,| op supe, :Aueysng je Jowelg ool 


“QUUSIAOYICUI asayjodAy ,| & SUOTORIJUI Sap SIA-B-SIA UOTBO 
-IJISSP]D ap INALA.| oP WOT SUONUINSS sap assoysnqgol 
RI JaNsnji,p ule uone[nus ap spnje ouNn.p sjelnsel 
sap suojuasaid snou ‘9"¢ WOTIDas BI] YW “SNOSsap-1d 7’ ¢ UOT}DES 
PI B SAILIO9P SopouIeUI Sap opre.| & WOT SuoNeUINse sap 
Z[eQO|S SIpIeA eI onTeag yOINId suoAk SNON “WOTW PI oP 
asamodAy ,] JUSUIA]OAIIP Ja1LIaA Inod gsodxe juasaid a] suep 
JasI[NN 919 BU SAPOYJOU Sad ap aNNe | (U suUN | IN ‘sdus} ap 
apoued onbeys nod MarIAIJUISI ap SAQUUOP ap 39 SANIL .p 
uonenyIs ev] Ins Jaued odAy ap saguuOp ap UOsSTeUIqUIOS 
oun e jadde wey (2661) oUIoyesueT] 19 [Og op UeA 
Jed ogias3ns ‘apoyjeul anne sug (7) Ud UN sIPIO.p 
UOTOLYSAl PT & Z SIPIO .P QUUSIAOYICU! UOTOLNSAI ouN oNjIs 
-qns nb jaued un,p ueAoul ne sarjj[iandel saguuop ap Sadho 
anenb Ins sapuoj spoyjsW suN sJasInNs JO (986) MNT 
op 19 [070 ap ue, ‘Joued un.p uaXour ne saljjlansel soguuop 
sop inod suuataoyseur aseyjodAy,| ap supryea e] op uo 
-ENJPAY, P SOPOUISUI XNOP SUIOU NE d}STXd [J “UONLOIJISse[D 
ap sInaLie.p soueseid ev] ap asned e onuUOdUI jsa IneTd 
-UI® | JUOP UOISIO}sIP duN JUaLIOdUIOD saguuop se] anb sored 
SOQAIQSGO S9QUUOP Sap apre | & JUSWa]qeUSAUOD sgUTUIeXO 
ang sed jnad ou asayjodAy 39909 ap suplyeA P| ‘stojajnoL, 

: ‘ro[dua .p 1asueyo 
op UleI] US SAUUOISEd9O SINdUIQYS ap UOTOdoOId ayo} snjd 
aun yuaulatqeqoid ajduros (| apoued ey ap sed uou stew 
Z apousd ej ap sinoo ne 1ojdura sues) adnoisd ad “| apoused 
®[ ap sINOd ne uoU sTeul Z apouiad ke] ap sInod ne 1o;dwe 
sues souuosiad ap adnois a] anb sanbruosys sinsuigyd op 
uomodoid ayo} snjd oun juowlelqeqoid aqojsue | aporied 
PI Op 38 Z apouied ev] ap sinoo ne 1ojdura sues souuosied op 
adnoi3 x] “| apoued ev] ap sinoo ne 1ojduia sues JuoweTese 
JUSTe}2 sayja,.nb guuop jueye ‘¢ spouted e] ap sinoo ne 1oydwie 
sues O19, p saouRYd ap snd juareae Z opouiad | ap sinoo ne 
1ojduia sues souuosiad sap ts ‘ayduiaxe sed ‘ayurasyus j1eJEs 
aseyjodAy,J “afjansucur gyANOe.p uoNenys ey] Inod 2]q 
" -eyea sed ang-jnad sau ouuatAoyIeU aseyjodAy | sjanbsey 
SUEP SOLIBUSIS ap dIQUIOU UTe}LIS0 UN IIOAVOU0D Jned UG 

“ajuapsogid 
apoused ev] ap yuewasueyo np sed pusadgap ou ‘ajuapgogid 
apoued e] ap sInod ne UOTENIS ek] ap nuaj ajduro9 ‘ayueI 
-NOd uOTeNyIs e] anb 3sa uOTeJeId19}UI aNe ouUE ‘eNnuUOD Z 
opoudd ev] ap sinoo ne uoNenyis e] soy oun ‘] apousd ev] ap 
sINOd ne uOTeNyIs e] ap sed puadap ou ouuosied oun .p uoN 
-ENUS d[qeILIZA L] ‘¢ apouiad e] ap sino nv nb ap-e-}sa 9 


(Z) A'3\zy, Ee Ax'S|2,, 


suosoddns snou ‘(Qg6|[ MNdT ap 19 [O”g op ura 
TOA) B[QPOUl NP sU[IqeIJMUSp!,]| Op sTessedgu UONIPUoS 
oUN ‘UN aIpsO .p sUUSTAOYIeUI ssaujodAy | op Naa Us ‘UU_ 


x‘BlA, B\x 


(1) Soins Se So u Ps flee 


u 

isa ¢ apouad ev] ap sinoo 
ne Z39 7 apouad ev] ap sinoo ne 4 ‘| apouad ey] ap sinoo ne x 
SIATIOV .P UONeNIs ouN sUep dANON as 3 adnols np aiquicwl 
un,nb syiqeqoid ey ((3=.9 ‘x= ‘k= 4 Iz=7)1g "2 
wm (8=9%x=x [d= pig “Mu (8=0)2=7'=4 


6S1 


‘x=y)ig ‘7'" “a yog ‘yuouroansedsal ¢ 10 Z sepouted 
sop sinoo ne °7 3a x nod sajqeyquies suonruryap sap dae 


| 2poudd ev] ap sinod ne 
SATJOVUT 4Sa (7 ‘3) QuUOSIOd EJ Is ¢ 
| epoudd e] ap sinoo ne 3 
1ojdurs sues 3sa (1 ‘3) ouuOsIad RI Is Z - 
| 2potied ev] ap sinoo ne 
seXojdure jso (2 “8) suuosied B[ Is | 


SUTUOS JULZSP 9S By nau mh ornaines 
‘, =81nod) 3 = 5 adnoiz np ouuosied owiai-7 ey Inod sy1An 
-OB | OP SIA-B-SIA UONENIS LP] SP SUONLOIJISSE]D SA[QRILIDA 
sgt ‘777 10 ‘x ‘"y MOG ‘ans op Isure 39 7 odnos8 I nod 
z = 'D ‘| odnoiz 9] suep danon as uoneindod er] ap aiquisul 
OUI a] IS | = 'D ‘Isury ‘adnois np uontsoduroo ve] ausisep 
inb afqeuea oun ‘5 310g ‘(a[dwiexe Jed ‘axas ‘aoel ‘ase) 7 
sadnois ua S$gdD BI ep e[qIo uoNe[ndod ev] suostaIp snoN 
“uores 
-I]e19Ua8 e] onb awWgUI ap suoNeor{dde sane .p suolessns 
Ud SNOU 19 ‘SgdD LI] Bp 3}x9}U09 J] sURP WOTW 22QPoul 2] 
SUOIJOQEIP SNOU ‘UOTIIES aJUasaid PB] SUR “a[QPOU ap saneuI 
-esed sap ayI[Iqeusd | op SUONIPUOD sane Pp JLOSp IO SII 
J2 ‘Q]QUIJUAPT Ise sJQpou 2] sajjanbsa] suep sUONIPUOD so] 
TIQeI9 IO (9861) MANET op Jo [Od op ueA (7861) Uss|Nog 
Jed sguuonsajied 19 (€/61[) sUIsSIAA Jed sgsodoid 39 
PJOQE ,P JUO JUDJL] DINIONs Op SUSTAOYICUI SSTQPOU Sa] 


SdNGAL Ad 
SadoOmdad SIOUL YNOd ALNALVT 
FANLONUALS AG ANNAUIAOMUAVAW ASATVNV “7 


“QUATO® | Op SIA-B-SIA UOT] 
-ENUS BP] SPP UONLOIJISSL[D Op INDIA | op suoNeNyeag sainjnj 
ap nod WOTW epoyiew e] ap oy1[9N,| op jafns ne suonep 
-UPUIUI09AI SOP SUO[NULIOJ 39 S}e][NSeI SOU SUOLUNsgI snou 
“¢ UOIDAS PY B ‘ULJUA “MOTAIOJUIOI-MOIAIOIUI SSATVUL dI[OA 
-NOU dUN,P SUONEUWIT}Sa xNe 19-Sa]]a0 JURIedUIOD US JUSTE} 
-OU ‘YO TIW SuOneUINse sop siIPI[eA Be] SUOSeJ Sa]UDIOJJIP 
OP SUOTJLIDA SNOU ‘p UOTJOAS BP] YW “"WOIW e[2Ppoul Ina][Iour 
NP UONIUO] Us UONRIIISse]D ap InaLe,; ep suoNRUINse 
sap juowlsesa suosedgid snou ‘uonses ajuasgid ey sued 
“SQ]QPOUI so op JUoWA}sN[e | JURUTLXS UD JNO} SgD PI ke 
S2]QPOUl op aLigs oun suojsnfe snou ja ‘Sg_ eI B eanbrdde 
VOIW epouleul ek] ep uoNsanb jsa [I “¢ UoNdes eT YW “sanbu 
-09U} SJUSUSPUO] sas 39 UONLUINSA,p epoyieu eT onb rsure 
VOI 2[2poul 2] SUOALIDgp snou ‘a}URAINS UOT}DES BT VY 
“UOTRIIJISSED 
OP INILI9,| IP SUOTL[PLIOS sap Ja asned eR] ap jafns ne sasay} 
-odAY Sd] JOUTWUXd PP 19 UONROTJISSeID Op INA .p Sayapoul 
Op I QvJapIsuod SUTUS OUN IOIJLIQA Op aysATeur | & JoWLIEd 
[A1dIZO] 3d ‘aylTeIQUas BS B 19 ASsaTdNOs Bs B BdRIH *S9Ud}R] 
S2|QeLIVA Op doUSsSqe.| US NO sdUasgid US sanbrUYWesO] 
SdITRDUT] SO[QPOU! Op assed opuvIS ouN Jajsnfe .p joued [1 19 


000Z eiquisoep ‘a}enbus,p senbiuyoe |, 


‘(L661 JUNULIAA ) WA) 1S9 osATeUL aNOU SUEp asITNN [a1OISo] 
J] ‘SIINOS SaJUDIOJJIP 9p Sa[qIssadov JUOs aNNe NO WOTW 
odA} op a]Ua}e] DINJONIYS Bp sa[epoul ap xtoyo uN 4no} 39} 
-snfe & JULAIAS S[IIDISO] SFT ‘SD B] OP oped 9] suep sanjoe 
uone[ndod e] ap uoNeoljisse[d op Inala] ap uOTENTeAg,| 
Inod YOTW uonesl[gpoul op sisajens e] op syn] 
9p 389 Inb 39 Inod sje}[NsgI sou WLIDep gsodxe yuasaud a] 
‘JULUDIUTLUI So[qepsoge ang-jned yuos saguUOp ap 
enbuew un.p osned e juosgid e nbsnf sounuexa .p ajqissod 
sed 31e9,uU [I,nb joued sed saygnbue sap saguuop sap sijenb 
PR] op sjoodse sureyso “uonbgsuod Jeg “MIIAIOIUII eB] Inod 
Nusjol Hod JUSWIOATLIOI UOT[HULYDE,| B IOIUIT] 9s ap Not] 
Ne UONLIJISSe[D Op INS .| op sUONRUIT}sa xnv Joued uN .p 
USAOUI Ne Sal[[Iande1 segUUOP sap s[qUIaSUa | JaIZ9\UI,p 
SUTIGIssod eI Iso WOTW osareNs P| op osejULAR one UA, 
‘(L661 Suroyosur’ 12 [Od ap ueA) 
UONVIJIsse[O Op Inawe., op yofns ne oyou snd s1ooUa 
UONULIOJUL OUN,P UONUI}IQO,| OP ONA US MOIAIOJUI9I Op 
soguuop sep jo joued un,p udAOUI ne sal|[ionoes saguUOp sap 
UOSTBUIQUIOD Fe] Inod snbysnejs ewgyds UN WUWINOJ WOTW 
a1sgiens ey ‘snd oq ‘SMOIAIOIUIOI Sop asATeUR Pp SaTTauUON 
-Ipen seseujodAy soy JOILI9A & 39 JOST[apoul e IAJos ynod 
VO'IW epouseur e] ‘asATeue ajuasaid ve] suep OWIWIOD ‘IsuTY 
“UONRIIJISSE[D OP INdUId,[ op UOHeNeag.p onbndAjeue o13 
-92I)S IN 9UN JUITJJO WOT 19 MOIAIOUISI Op sopoyjeul 
Sd] ‘SoTqIUOdSIP JUOS MIIAIOJUIOI Op saguuop sap saljanbse] 
Inod ‘sq ke] aumu0d jaued sed sajgnbusa sap 3sa nb 90 inog 
‘saygnbus sao op aINsoul op INdLI9,| Jon[eag,p UOdSeJ anaes 
eR] ang-yned 3sd WOT aiselens ey ‘osuodal ap neapiej 
Np 39 sovjd Ins xneaey sap 9yIxa,dwioo ev] ap ‘saitejespnq 
SUOT|OLSI Op asned eB sazqissod sed jUOS dU SMOIAIOIUIOI 
sg] ‘joued red sajgnbua sasnaiquiou op sueq ‘ajgnbua.p 
soguuop sop syyenb e] ep uonenyead,] op iso Inb 99 
Inod YUIMISEH Jd ITe[OUIS Jo ‘SIOWIUUNS 39 BQID}O ‘IOUT[9Z, 
19 pPMOQY “Ja][NJ 39 BNYD ap sopoylsul xne 39 snsusd_d 9} 
Jo nealing np sajjauuonipey sapoyjow xne yoddei sed sasey 
-UPAR SUILLI9O dIJJO SPOUIOU PI ‘pIed9 199 YW “MOIAIOJUIOI 
9p SapoyIeU op apre,| B aNUS}GO aINsoU oUNONe IU sUIAIx9 
I[QI[LeJUL INSU SUNONL 93IX9 ,U d[]9 ‘MOIAIOIUI Pp SOQUUOP 
SOP JUOUIDJIOIIP UONPOIJISSe[D OP INI ,| Ins SUOTJBULIOJ 
-UI,p UOT}DeINX9 | ap ana Ua oued sed saygnbua sap sanned 
-91 DINLU Ll ap aBeULAR IN WOT ssores ev] onbsing 
“PUAN | OP STA-B-STA 
BPAIOSGO 19 BIQUILIQA UOTJENIIS Be] e WoOddel Jed syed soy 
guesgidal Inb UONROIJISse]O ap INS ,.p s[Qpoul uN Jo ‘ayIAN 
-O8 | OP SIA-B-SIA UONLNIIS B[QUILISA LP] Op o1INe | B STOW UN .p 
SJUSUIASULYD Se] aJUasgidal INb ajUS}e] SUUSIAOYIeU oUTeYyO 
OP I[Qpoul uN :safepoul xnep op UOSTeUIqUIOD 9UN 9}I[e9I 
Ud ISO WOT POU! VY] “MIIAIJOIUL,P SOQUUOP Sap USUI} 
-SOIIP UONLIJISSe[OD OP INALI,| INS UOTCULIOJU! | ap ste] 
-xo,p ule aued red saygnbua sap sannedes oinjeu eb] op 338} 
-UPAP dJI] DPOUIPW 3NBD “SddD eI & eenbrjdde 919 s109ue sed 
wWeae.u ‘joued sed ajgnbus,p saguuop Sep UONedIJISse[D op 
INIUI | xp UONeUINSE p ssnayoWOId sI1d9}eNs 9UN *9}UI}L] 
OINjONNS sp UUSTAOYIeUI ssATeUL.| VP VpoysW eT 


‘9SI[NN sfgpow np saseujodAy 
Sop aJg[dui09 uoNeoIZIGA ouN sed juonouLIEd su AayD0YS 
OP S9QUUOP SII ‘YUIMISED 39 IeOUTS Inod suMWIOD ‘juaWIaS 
“hoINoy[eYl “a[Qpoul ne Hp sieiq un ayg-jned onbrput tnb 30 
‘sInoyne sojne .p Jed sg[eusis xnv} say] aonb sgazqe snd dnoo 
-heoq jUate}9 INSLII.p xne} sag ‘suoneULye sas 1oAndde 
Inod sinajoey Jed ash[eur .p Saslo}eULIJUO sapoyIeul sap Ins 
1QINId JULPUO]J Vs ‘MaIAIO\UIOI Bp SaguUOp ap gstMN sed v.u 
AdYOOYS “MOIAIOJUL,] OP OIPed | suep a9IIOsns ssuodgi op 
InaLa | Jed gysned ong-jnod ysa (¢/.6[) sepleg sed psoge,p 3] 
-BUSIS JUSWA][9ANOUAI op sodnois xne Op steiq a] anb gnbip 
-UI & aSATeUR UOS “Sd PI & Ua}e] sINIONN)s op asATeue oun 
anbiydde juswieyes9 & (9861) Aoyooys ‘uaUIaT[aANoUal op 
sodnoi3 xne Op sieiq np sjueulUajgp sap apnig oun sueq 
‘SdD P| 9p Seguuop se] Inod sayqe[eA JUOsS a[Qpour np yafns 
ne sasgujodAy sao Is yuowia}9011p sed onbiput ,u ascyeue Ing] 
‘juonbgsuod Jeg ‘a[2pouwl np Juowiajsnfe .p onbuew np uon 
-BOIJLIZA P| Inod s[qruodsip jso,U [aNpIssl 9uaqt| ep aiZep 
unone ‘sanguieied sep uoneultysa | Inod sayqruodsip suaqiy 
OP SQIZap Sa] Sno} JUdQIOSge s[epoUl ep seeuIEIed sa] aonb 
-sIng ‘sodnois sao Inod sjuaigjjip Uos satoe UONe[ndod ey] 
OP SANOR Pp xne} so] onb sipur} ‘sawUW9j soy Ja SOUTWOY So] 
Inod sayes9 JUOS UOTLOIJISSe[D OP INIA ,p sazt[Iqeqoid sa] 
onb gsoddns uo yyIMsed jo IIepours ‘axas a] UOTas segdnoo 
-O1 MOTAIOUI9I-MIIAIOIUL P S9QUUOP op apIe_] VY “‘sopeoIpeul 
sanbusouserp saaneidg sap inod (9g61) 12178, 19 INF Jed 
JUSUIITBUISUIO sagsodoid a,epoul ap SUONOLNSaI ap UOTIDUOJ 
Ud SdD B] OP MIIAIOUIDI-MOIAIOIUL Pp SOQUUOP xne 9]U9}e] 
OSSP[O OP UONeSI[Spoul ap aIsa}e.Ns oun snbiydde yuo (9661) 
YUIMISeD 39 ILPOUIS 19 (9661) YNIMISeH Jo IepOUIg 
“Sogstelq 
JUSIOS JaT[N 19 ENYD ap aisgeNs el op spre. R SIgUITISO 
asuodal ap saitiqeqoid saj anb ynad as {I “uanbgsuod Jeg 
‘(S661 O8Y 30 YBUIS ‘1661 YsIeuesyoINy .C a[duroxo sed 
JOA) SgD kB] Inod g{qeea sed ang-jned jsa,u aseuiodAy, | 
anb juenbiput syuaumnoop sureyiao onbsind asdyeur Ins] op 
ay] oun aJuasgidal YD] eseuodAy J ‘uoT[NUeYdE | suep 
anne | & SIOW! UN.p ja (UONRJUDUNOOp eB] SURP YO] sso) 
-odAy ,p pied uo) MAIAIOVUIAI BI 19 MOTAIONUT,| Inod sayuep 
-uadgput uoneoijisse]9 ap sinaiia sop juasoddns sit ‘snjd oq 
*MOIAIJOUTAI BT Inod aonb MarAIOIUT,| Inod jue} [NU ang B UOT 
-BOIJISSL]O Op INILA,| & UP steiq a] JUeSIOJ ‘esuodgI ap SOIT] 
-Iqeqold xne suonoLsel sasnoinosi ap yussoduit sinoyne 
S90 ‘a]Qpow! Np sITIqeIFHUEp!.P Tonos Jeg “SO eI e esuod 
-91 ap sailjiqeqoid say JowINsa.p ule SqD eB ep sgyooiddes 
MOIAIOUISI Op S9QUUOP xne dJUAIL] SINJONIYS op s[epoul ap 
ad4q un gnbrjdde quo (¢g6T) enyD 12 JoT[N 19 (L861) OLIN 
19 ENYS ‘SUTIQI[reJur.p esemodAy | op IMNsqns swWWOD 
"p661 U2 SdD PI OP MATAIJOJUIOI Op oUTUTeISOId 
np saysoiddes MOIAIOIUIAI JO[OA NP snsusD sy} Jo nesing 
g[ Jed uoneunut[s,| op ajqesuodsal 939 & adUaTNSUOD ap 
asud aneo ‘ail[eel uq “so[qeIsoidde uONeoIfIsse[D ap SINoLIS 
sap & Juaigid 9s MOIAIOJUI9I Op SeguUOp sa] enb suoneoIpul 
sojueyoduit .p juajuasgid sinayne sap ‘sanbipLiga sino[eA 
sop juasstuinoy sagydoiddel sMoIAIOIUIZI seq affenbe| 


@]UE}e| GiNjonjs ap uUaIAOHJeW esAjeue,| ap sypIeA :ALeysng je JEWS! gS 


Da ‘uo SuIyse pA ‘sousHeIg UONeYOdsuRIL Jo neaing ‘AJaysng ‘I UYOLS 19 6OLLZ ON “Wed BSuCLLL, Yoreasay ‘ayn Hsu] s[sueLL], YoIeasoy ‘IoUIOIg “g [Neg 


uOTas asayIOdAY | BINOP Ud STU UO (QR JOUTOIYDS {166 
JOUIDIYIS 39 ULUISIO, ‘76G |] ULUISIO,J 19 JOU ‘8661 ‘9661 
YUIMISeH 39 Iepoulg afduwioxea sed) sinoyne sInoisn{d 
‘g]Qe[QuIas aISaIeNs 9uN a}dope 
1UO (SQ6]) JOUT[AZ 19 PMOGY ‘senbIpLiga sUIWOD JOIOp 
-Isuod jnad uo] anb saguuop sap IUINO} Sgd PI OP SMITA 
-Io\UI91 Sop yusWayso01ddel ap snssad01d a] anb asaujodAy,, | 
INS SPUO] as sInajne sad ap ajUad—I syd asATeUr | ‘O86 OP 
JUSUINIOP 9] SUBP SUIWOD “SANL,]| OP SIA-B-SIA UOTJENIS 
op JUSWOSULYD OP xNe} So] INS UONPOIJISSe[D Op INIA] 
OP 19JJO,| JaN[eAY,P 19 SD PL] SP UOTPOIJISse[D Op INd1Ia .p 
XN} Sd] JOWINSS Pp Ue SGD PI OP MIIAIOUIOI Op OUTWURIS 
-o1d np saguuop sap asi[yN 1UO (GGG) SIOWIUUNS Jo VqIB}0g 
‘WUSWISIOY “(C861 JOUTIEZ 19 PMOQY ‘O86 SISWUINS 
39 BQIDNI0g £/ 86 JOT[N-] 19 ENYD {766] UPUWSIO, 19 IOWIDIg 
‘8661 ‘9661 YUIMISeD 39 IIe[OUTS IIOA) UONROIJISSe[D Op INdI 
-19 | JOWNSS Ud,p Ue SqD P| Op MIIAIOUIOI Op ouTWIeIBOId 
np saguuop se] asATeue UO Snsud_D oY) JO NeaIng Np INdLI9} 
-X9 | @ SINdYyoIOYO sap sed saredgid syuswiNdop sinaisn{g 
‘SdD BI OP MOIAJOIUIEI Op oUTUIeIZ0Id np 
ag|[leyap UONdLOsap oun (166) JoUTaIYOS 39 UeUSIO,] suep 
PIDANO] UC ‘(Sesuodal sap BdUdIZYOO kL] Op NO) 9II]IGeIS 
BR] Op uoeUIss,| @ Was Jo INY.pinofne alooUs jINsInod 
as s9ydo0iddes UOU MOIAIOIUIAI LI ‘SIOJaINO], ‘saguUOP sap 
guyenb ey op yafns ne suonednooogid ap asneo e p66] Ud 
QUILUT[S 919 B ‘SMOTAIOIUI Sap outTenb eT ap 9JO.NUOD Np ona ud 
jnouns gsi[MNn ‘saydoiddes MITAIOIUIOI JOTOA JT “sgyooiddes 
MOLAIOJUII BUN B $}UR}SAI seNb stoN sa] 39 saydoiddes vou 


“‘V'S'N 06S07 
I 


MOIAIOUISI BUN & UOT[HULYOY,| op UOITAUS YeNb UN sTuNOS 
e UO ‘p66] Ua .nbsne ‘sae UONe[Ndod x] Ins suonsenb 
SOP JUSWIWULIOU ‘BTPUISLIO MOIAIOJUI | Op SuOsoNnb soureyia9 
nesanou op asod uo ja SgD Pe] ap sjuepuodgl sap (%¢ 
op SUIOW) UOT]HURYDE-snos Jed UN aI1} UO ‘oUUeIZOId 99 
Op SIped 9] sue “daT}Oe UOT}e[Ndod Fe] Ins saguUOPp Sop sITT 
-enb vy] Jan[eag.p ulye Sgd P| op MOIAIOVUIOI op suTUeIZOId 
J] SSLIIP B sNsudD oy} JO neaIng 2] ‘OGG SeguUR sop Ingep 
gt stndop ‘tsury ‘sastidas sinorsnid e sonyeag 939 B SogUUOp 
Sop opnyyoexa | ‘ayeyuowouIaAno3 onbiyI[od ey] inod gga 
PR] Op SeguUOPp ap SaLigs sop soueyoduil | ap nua} a}durog 

“oatjoe uole[ndod e] op ayjonuue 
19 O[[ONSUSUT UOT|NJOAY, | SIAINS Inod §q_ | op saguuOp op 
soligs xne jodde yej a[elopay UONeSIUTUUpR | ‘Zp/6] SIndap 
‘sKed np sgjo sonbrurouodg sinayeoIpul sap UN] JUOS Sd PI 
ap soi satjoe uoNe[ndod vy] ap suoneuNse say] “sIuA-sIeIq 
sop sSurmeq pue juowXojdurq ut sonsneig soqeq] jo 
neaing 9] Jed stout onbeyo saegijqnd yuos sanoe uonetndod 
PR] Op uONIsodwi0d ve] op UONNTOAY,| ap 39 UOTIsOdWwo09 
Rl Op ‘ayjie) ve] op isa Inb 90 inod Sg_ BI op seein sofeuoneu 
SUOIJBUIT}SA Sq] ‘SIUC)-s}eyq Sep sae UONe[ndod x] op 
sonbns1ig}aesed sarne .p Jo S8euQYS np ‘to[dure,] ap suon 
-eull}so pp UONeIedgid e] ap ona Ud sIUP)-S}eIq SOp snsudd 
oy) Jo neaing 9] Jed so8eusur sap soidne yuouayjonsueur 
SQUdU UOT[NULYSE Ins a3gnbus sun jsa (BATJOV UOTe[Ndod 
eR] Ins ajgnbus ‘sqoa) Aesaing uoneindog yuouND ey 


NOILONGOUINI T 


‘saguuop sap 9itjenb ‘adewoyo ‘eseuuor[HueYo9 | R ONp UOU Inoue ‘joued Jed sajgnbuq = :sq1D SLOW 


“QUpljea op saanaidg buto say inod sje}[nsgi suog op QUUOP & 9POIU BT “WOT IIS9IENS P| Op SIIPITeA e] OINOp Us oI}0W 
op UOsTeI OUNONe }IUINO} ou asATeuL dJUISgId eT ‘QUUDIAOYIEU BsQuIOdAY ,[ B SUOHIIJUL.P PIeSd,| W VOT SuOHeUINss sop 
QSS9}SNQOl BP] JUDUTUILXO sINayNeK so] ‘sn{d aq ‘UONROTJISse[O Op INdIJ9,p s[yod sap azt[IqQisneyd (9 39 oUsIgYOO op onbueul 
NP 1S9}9J-}S9} MOTAIOUIOI 9dA} op 19 ofapour adA} ap suONeUINSA So] aU PJODOR (p ‘asuodal ap seit[IQeqold sap ysa}aI-3S9} 
MOIAIOIUIOI 3dA} ap ja B[Qpou adAj ap suONeUINSA Sa] aIUA PIODIe (9 ‘SGD PI AP Soguue sion Inod sapoul np yusua\snfe | op 
SPA (q ‘sgslJapoul sonsouselp (e -WOTW I8oIeNs LP] Op dIPI[LA e] Op suOIsuoUIp burs Juopsoge sinayne say “WOTWN 
JIB ]eNs LP] IP SIIPI[LA e] JOUTUIeXS B OpIe ‘WOTW Odi op suoneunsa xne oseduroo ey] uo Is ‘Inb ¢gD kB] ap UONLVOTIsseyo 
ap INdI9 | SP UONLUINSI,p s189}e1}s Ne 9UN JUdJUNS9IdOI MITAIOIUIOI Op SaguUOP sx] “sduIa) op sopoued sourgui soy sid 
nad e inod $gd ek] ap segydoiddes uou MaIAIOIUIOI op SaguUOPp sap oireyUsUIgTddns asATeue oUN QUOUT UO SII 19 (966 ‘S66I 
“€66[) SoguUR SIo]] op oUNDeYD ap slour sJotWeId sioy say Inod (aanoe uoNe[ndod ey ins aygnbua ‘Ss ga) Aegaing uoHe;ndog 
JUSLIND P| IP MIIAIJOIUI.P SoQUUOP sap gsATeUR JUO SINajNne say ‘soguUOP sop szITenb e] sp UOHENeAg,p so][ouUONIped) 
Sapoyjou sat soe{duo asAyeue ,p add} 99 anb gyI[IGIssod eT] JonTeag eB 39 VAIL UONLINdod ve] ap UONLOTJISse[O Op INdLO,| 
Op ( d]U9}L] SINjONI}S op DUUDTAOYIeU VsATeUL) WOT SUONCUINSS Sop dIIPITLA PT JOIUDA & QYoIJOYO JNoUNs JUO sInayne sa] 


aWnsay 


:AMHHSNE “W NHOfL 9 MAING “d TAVd 


9A398 UOL[Ndod vy] ANS saguUOp sap 
UONVIVISSE]D IP ANIAII | 9p UOVUIT}SI | Anod 
9}U9}U][ IANJINAJS IP DUUIIAOYAVUI VSATVUL | IP 9IPIYVA 


ZSt 


epeueg enbiysiyels 
LZL-ZSL “dd ‘Z ,u‘9z IOAN 
000Z eiqwadap ‘ajanbua,p sanbiuyoe | 


; pines FN St rs 
wea) pathran) 62 


ut a legal 
id tig oo mani priv 
4 aye i i ee Ay i : 


p - Seq) pas as | N-y) jie 
marty ahi yreer’s fen + 
hk oD Mh iS wee Sp aly 


A GAY 


a pee, n , 
ty waeeey aes 
ca na "6 a 
Sw Wee oO }ie We 
iy: ; 


e 
Am, > 
vk Wik 4 


i, eN is 

ail ans Niles 

hes} sions Ath POG) we Sas 
a a eves dati | Aired vei) th ; 


id re iy spat. hs ne 


<> 


ri 1 
4 
vat 
a 
é ' 


hey me se 
y 4 P : i f eal 
: i Hele dtte /ve Breit ail 


che " ald Me Va) F hs intl iA 


7 
MG 


pene gp ecieny Ae calenartyan. 
’ Quay + 


Sarees 

Rata iy) aL) fs jaca 
we Neb «cE \eeesmeitad smontss We ee ti 
os meas ie Fea ner: ee | om 


ul so ; q A i \ sae cy i 
a eae a oe : 


bod, de ae " - J 


ane wie MA f age 

wy nia” Sung . Sew 

Puniog PL resepAS | 
in ty 


PUT TAAL #52 
Ieiniy Sebo sy ie 


gvenaes a WeIeea 
1342907. FORO 165, *4 


: 2! 
f*t RO: 7 i‘ it 
i > oy ns > é L diy ' ber” ak ; 
4 i. art r } g erav) wt xf 
001 Base Ho fi ’ ’ foie ’ 
7 
4 a UWA, Ay ae ROE 24) : rac) ' SAse OF if i ny 
MP Ae The he er ae 
ity vk Ke Ver aA Pte 7 7 0 a a" ~»4 aA EL 
( ty 
rats Kyo ne = Any ee 
’ NPS Sei e, ape ane iy Pa 
ae “Oo (3 SS car + ; ey “yar 
+ |. IZA 
ate ar Ph oe) 
a ‘ ‘4 i é ‘ ’ 
* t ret - f i 
7 2 ewe ¥ j ¢ 
4 a Af ey ~ 
i) Sy 5 S ae 
Mh Lit Wee 4 
sy a2 ' gts j ‘ 
é A i , y . i : | Pare ~ \ Pes “ 
7) 4 « an hes t 
PVG Ark, ch Guatint dt. ME LANe Lhe 


3 {wa tae es arte 
nom x PPR! Acre fp, amet YAN wns WA Cavchst 
oy 


ior’ Liaw ee ym eg hotly Gem aoe ane 


; “et 7 aro 
edyrt.c Fee). oye Gia ro ates ep eiteeges Tales apr 
SHEE OO. HOC TT Lot oe gen Tani (o> ate Qe eno. ae, 
ey Che 2° sePRrER OR OE Tata ek ee ee ag 
Pe Y . Gh tp? Ve i. vay 7 


gana 3 ous 


“Sei wh &- Oak p ‘ ‘ Ve Ten wee 9, 


wih ee Lue 


“‘Be[JOA-JasuUdS :yIOX MON “sUu1jdupng Aaaing pajsissy japoy 
(7661) “f ‘NWALLAYM 3° “d@ ‘NOSSNAMS “d-'D “TVONUYS 


‘O0Z-E81 ‘7 ‘8 ‘sousuDIS 
pw12yfo fo pouanor *'g yenboun soy Sunysieq (7661) “TI ‘HSI 


“AQTIAA SOK MON “BUldung Aaaing *($961) “1 ‘HSI 


‘OZL-61I ‘I ‘Sz ‘atanbua,p 
sanbiuysaJ, ‘addes3 ap ja}J9,| & 19 UOeIgpUOd Fe] e SOT] adepuoS 
ap ued ap sjajjJo say 1nod ysry ap s[NuULOj ev] ap sJgpoul ap aseq 
g uoneoynsne (6661) ‘d ‘TYIHVT 19 “S ‘YAGAVH “S “YATAVD 
“KOTEM 
YIOX MON “P92 .€ ‘Sanbiuyoay Buyduvg (L161) ‘DM ‘NVYHOOO 


HIHd VUIOITEIa 


SSI 


I=! 
“o°0" "das 'm'3 K 
N 
10 
MN ‘94 ‘d ET) oro" Cd= 
I=! 
3 Manan M3 a pelt 
OMN + "9° 9" TAT = "M3 cx 
N 
anb suojou snou ‘(¢) 11u9}q0 .p UY 


0002 aiqwesap ‘ajenbusa,p senbiuyoe | 


| I=! US? 
M's “(07 + '3 C+ [base 'm¢ j>- (QA 
N N N 
ouuop 4 Jed JOsIAIp ap 19 (6) ap (g) aaleNsnos op 11ej oT] 


I=! I=! {=! 
ayy Cie. ! 
(6) “3 sees gon + He he oe 


[=! I=! 


NE d/: 2 hed, od 4 


I=! 


[sz +'g/'307 + JZ +'d/{3 + 'd7 + 'd/2] <e ‘d/ ua 


anb je 


(8) aS ta Gonz a2 2O2N = ad + DN) = 


anb asoddns ({) enb suojou snou ‘ayinsuq 
he tai Oana u= (J) SuoAe snou (Z) op med y 


(¢) JONVINVA VT 
ad AINWUOA VI AG TANOITVO °s 


“(PSI 
‘766 UCUNIDIAA 39 UOSSUDMS ‘[epUreS) 1 911UN | Inod ayeqo[3 
YOHDe[9s ep siiqeqoid ey soy ,; u awWWIOd 9gIgpIsuod 
queria ‘yg “(Z) Us soURLIeA eT SIOy (A/U-]) JUOWOANLU 
-1xoidde jso [810] np uosduoYy [-ZAIOH op InayeuNysa | op 
OURLIA FP] IS OSTUIOI SUBS SQIIUN U ap SaTeSgUI saI[Iqeqoid 
B gisop UN & SUOT[NUeYDE sop e yuonbi{dde s asepuos op 
ueld op 39JJ2,| 1nod sg[nojeo suoae snou onb syeq[Nsad SIT 
‘(N/U - [) JUoWOAeUITXOIdde jso asTWIsI 
DOAe VSeUUOT[MUeYSES UN Inod JoURLILA LI 39 DSTO SULS 
g[duIs aojegTe aseuUO][WUeYSE UN Inod UOT[HURYDE,| OP 
QUUSAOUI OUN ,p ddURLIVA eB] aUS Wodde: a] onb suojaddey 
“g[loej 3so ‘amuoo Jed ‘syey[nso1 sop onbnysuney yUOUWISs 
-SISIe[9 UA) “soyUN,p sorted sap inod sagsoduwi0s uoNseT9s 
ap soiiqeqoid sap ajduioo suuen uo, aonb jresosoddns 
[E 1eo ‘axeajduioo snjd jresas astuel sues oseuUO][NURYd9 
un Inod j0ex9 asepuos op uvyd ap 39JJ9,| Op [No[Vo oT 


GSINAY SNVS FOVNNOTILLINVHO"U ‘b 


‘Inu onbsaid 
“g/'9 anod UST Op oJ op oyooidde os UONeUITSS BN9_) 


(L) “(£9 /¢) Mad + MAL + | 


@ JINpal as osepuos sp ur|d op J9JJ9,] op UONLUINSE | 
‘(C661 YSTY) «presey ne uoneigpuod» ey] ap sed 9] 3S Inb 
‘o = ¢°d suoxyy snou ts onb ‘qe1oeds seo op ont) B ‘suNION 


(9) (mad) (“9/9) + (Mae + 1)(4 - 1) 


“IMs SUIIOD UOTTHURYSE | op InJed 
R guUITISa ang ined esepuos op urd ap JajJa | ‘Isury ‘NIMU 
J1u9}q0 Pp youllad snou ,(u/A) sed InoyerguINU Np soUeI 
-9dso ,[ JOSIAIP op weJ FT *,(u/N) Op INoeUINse uN aUTUIOD 
9J9pIsuod ag nad (¢) op InoJeuTWOUYp 2] DUOP 19 ‘U/AV 189 
M op souvigdsa,T “u/YN 189 INeyeI9UINU Np aduLIEdsa J 


(S) ae 


onb pioge ,p suojou 
‘9puoj-uslq 9] cIpucIdWOd Us .p UJ ‘N/Mu JouNsa Inod 
MAL + [ JOSI[N suoAnod snou ‘a[qruodsip 3s9 UOTTHUeUDIE | 
onbsio, osepuos op ued ap jojjo] Jownsa.p uly 


HOVGNOS Ad NV Td Ad 
LaAdAa<T Ad NOILVWILSA °€ 


‘Issne xno sjnu juoreses “°d 
yo “2d onb auios op ‘g ap suoNauo] soy 19 3 ap suONOUOJ 
SI] dU I[NU UONR[ESLIOD oUN sUOLINe snou ‘aouRpued 
-9pUL .p JUSWIZTeS9 sIeU J 19 3 ANUS B][NU UONe[aLIOS ouN ,_p 
jUdWZ[Nes UOU sdUasgId Ud_Nb ‘stoJayno} ‘I9}0U Y “d/[ M 
ins anb joinjd g ans 4 assaxZau Inb afepour uN suep sTeULIOUe 
Ig|quies ynad soyqeeSt[Zeu juatos ““d yo “7d onb yuejNoa 
uOTIpUOd P| onb Jonbrewlal ej B INDJVUTUILX9 UL) ‘pULIS 3SO 
*o anbsio] a[qeyeA a1Sal [9 19 ‘satquasi[Z9u JUSIOS UOISsoIs9I 
B] Op snpisei soy onb sed os1xa,u uoneutxoidde 33399 


(r) “(-NIM),("9/0) + NiMUES- 1) = OP 


USWOAeUITXOIdde so asepuos op 
ued ap 19439] ‘sejqveSi[3eu quos “°d 30 “?d 1s quanbgsuos 
Jeg 2, Nj 4 919 Weine soueLIvA eT ‘asTUIeI SeAv [dus 
antoware aseuUoT[NUeYE UN B SINOdDeI No Weae UO] IS 
MIN 79(4°d- 1) + (U/N- MNO = (4)A JouUOp z UOSes 
op aarjt{duuis 189 (¢) Ua aouRLTeA FI ‘synU quatos “°d 39 “2d 
anb inod aisnfe uarq Zasse 3S9 UOISSOIS9I OP S[QPOU 9] IS 

*(¢ UOT}DIS BY B 

[No]e JJ IIA) UOISsaISEI op o[Qpou np JusWe\snIe | op yolns 
ne ssgyjodAy ounone Ins epuoj os ou uorssoidxa aap 

N/a" <= M 0 


(€) M929" 3dar nz + 


“o®o" dy + MN69(7%d- 1) + (MIN- MINZ0 = (QA 
JINS SUIWIOD doURLIvA eb] JawWLIdxegi suoanod 
snou ‘({) Uorssal891 op UONe[NULOJ e[ op spre, VY 


I=! 


(Z) A df) ad Gat =A 
N 


e;eBaul uolesapuod un snod snewixoidde eBepuos ap ued ap jojjo UN) «sa0UEdS vSl 


"W'S'N ‘80Z09 TT] ‘UoIsURAg ‘A}ISIOATUA) WD}JSAMYLION ‘peoy UepLIaYsS 9007 ‘Yoreasay AdI[Og JOJ aININSUT pue soNsNeIS Jo juoUIEdag ‘Is9Uadg ‘q sonIg ; 


(EST “LL6OT URIYSOD) 
onuuod UaIq Isa { OP OOULLIVA VT “{ ‘uoneindod ev] ap 
1210) np jamiqey sigpuod snayeumnsa ‘A’! me & "og 

‘d 10 9 ‘g ‘g sed ‘ojduroxo sed ‘d 19 ‘0 ‘g “0 JOWNSA 
19 ([) uorssel3e1 op uoNRussgidar P| eee suoanod 
snou ‘sajqiuodsip juos saguuop sap IS ° oe iu 8 79 
anb 19 Q = N/'3 Oe peda ne anb f° os “p 
ap SUOTIIUTJEp sop oJUaTeAINbs uosey 2p no ‘Sod SoIPUIOU 
a! sajgidoid sap ynsuo, S OT 2d sed m 19 73 emu 

“34 med mya 3 anuo ‘7d med g 19 anua uonvindod ap 
wone(gu¥09 1 ano SNON ‘N/A - tA) ee 9 ‘ajduiaxe 
red ‘sane “o 19 “0‘%0““o red m sap 12 23 Sop 4 sop ‘d sap 
uonetndod op soUvLIeA Soy Suo}OU SnON “Ny a! Ke nt 


RB (d-'d) Xd -'d) (4-'O' = 9 N/D- X = DO 
(1) aaa g = 0 =A 


SQUB9 SdIPUIOW Sop 
UOISSAIB91 OP SIOIP BI SUOIZPISUOD *,_ N = N/ ye Ke d 
enb Jaj0u Y “NSIS ‘N 2[ie1 ap uone[ndod dun Diet 
9u1un | Inod a8euuoynueyse,p sprod 2] “(‘gu)/] = 'M 10 ‘U 
Q][1e1 op UoTTMUeYDE UN Inod (anne | F a3eIN UN Pp) UONdETES 
ap suyiqeqoid ey “g ‘IgqislUI,p oimsour ef “A 10S 


NOILVWTINdOd 
V1 dd Ld FOVGNOS Ad NV'Id 
Nd NOISSHADAY AG NOILVINASAAdAA °7 


‘p UOTIOIS BI B SpJOE }S9 BSTUIOI SURS BBeU 

-UOT[HULYDE | & s}eI[Nsel sap onbysiunsy JUSWIESsSISIE[S “TJ 

“QSTUIOI DOAR SoTeSOUI soy]{Iqeqoid Be sIZEep uN ev BBeUUO] 
-[QUeYd9 | B SINODAI SUOAR SNoOU ‘9}10I[dUIIS ap IONOs Jeg 

‘yQ19}UI,.p UoNendod ey] ap 19 

’ gsepuos op urjd np sanbnsuaiovseo sap juswranbrun puedap 

uoneutxoidde | ap opninoexe J ‘uonejndodiadns oun.p 

9911] 1so9 UONe[Ndod ev] onb sed suosoddns ou snou ‘juesiej 

39 ‘uONnR[sLI09 oun OAR A ynod {I [onbo, suep s[epour 


un Inod osepuos op uvyd op jasjo,. op uoNneultxoidde oun 
101 suojuasgid snoN ‘(ags9puod uoNeWINsa | onb sinayjIow 
no ouUog Issne yno} Welos soigpuod uou uUONeUIT}sS 
oun nb oy0s ap) IQI9IUI,P I[quuUeA ke] 1a Ssplod say aUS 
UOTIe[a09 op sed v A,u [I fonbay suep ueyd un no sjapow un 
INs BpuoJ 9s asepuos ap uryd ap ja}Jo,] Inod mau + [| UOTeUT 
-1xoldde ‘J “a][nu assepoenul uolje[su0o oun.p soueseid 
Ud MAL + [ B INPS 9S ‘(GO6]) YSEY SuUep JURINSIy adepuos 
op ued un Ins sgpuoj sjeinsel sop e puodseios nb 
‘Q[NULIOJ INF] “jUsWaTe39 soddeis us ast ouN .p sduasoid 
ud asepuos op urd ap jajJo UN ITUa}Go Inod uoNe[ndodsedns 
ap J[Qpouwl uN ysI[VN UO (6661) YR] 30 JopoeH ‘“IaTqeyH 
“M/(m- "Hy" us mas ‘uoTHURYD9,| op euuesour RL S90 
a Is Jo ‘UOT[NURYDE,| op 7 911UN | ap splod 9] 1s9 ‘mM OUOp IS 
“UoT[NULYDE | op splod sop dANRIAI SOURLIRA P] SUIWUOD IUTJop 
1SO MAL NO ‘Mas + [ :SaTeBuUI sayIqeqold eB suoT[MUeYDE ,p 
$d9i1) sogigpuod suoljeuNss sop Inod asepuos op uryd op 
12JJ2 | op UoNeUIxoIdde oun qLWO9p & (7661 *S961) UST 
“Q0vdIJJo BSepuosS 
op urd un.p aonb adej9 ‘sadA} sinolie sap aiqeieoid 
uonoipeid ey nod aonb aqruodsip sa uo] WUeYdg,] anb stoj 
gun sadAj sina sop uoneutxoldde | 1nod ju) syueyoduit 
juOs osepuos op urd op sjojja say “(S961 YSTy) UOTIN 
-ULYD9 ,P d][Ie} OWIQUI P| 19 o[duIs oIOJegTe aseUUOT[HURYDE 
un Inod onbysne}s e] ap soueUeA ke] 39 UOT}Sanb Ud asepuUOS 
op uryd 9] Inod 3g19}uUI,.p onbysiyje}s ve] op soURLIvA Be] oI]UO 
wodde 9] 1S9 10-InJa_D “asepuos op ur]d ap J9jJ9,| Op opre | 
e salWuenb ang ined uonRUsWsNe sed “UOT[NHULYDE | 
SUPP DAISSIOXO UOSe} ap JUaTIeA spiod soa] Is sseuUOTIN 
-UbYOd Pp DOURLIVA RT Op UONeUOUISNe | Isa UONeIgpUuod 
op snssod0Jd Np JUSIUSAUODUI UL] “IQIDIUI,P B[QeleA PI 390 
aseuUo][HULYSE ,p splod say] aud UOTR[PLIOD oun v A [Is anb 
-I7RUI9}SAS INDO oUN suTeNUS splod sap aduasqe J onb sored 
Ue} I UO ‘UONdDITIS ap saz[Iqeqoid sap sasJOAUI SINsTeA 
sot Jed sajesout souypiqeqoid e uoT[MWUeYoS UN sUep SUOTIeA 
-IgsqoO So] JoJg9puod ke a}sIsuod ajueInod onbiyeid suQ 


NOILONGOULINI ‘T 


“soxo[duioo suoj[HueYyos ‘oseuUO][NUeYIE Pp soULLIvA ‘asepuos ap Uv] ap joyJa ‘uoNeJgpuog :SA TOD SLOW 


“‘uONe{gI0N 9uN_.p 9duUasgId e| Op a}dWI09 JtUD} Inod agjUasgid jso adepuoOs op uL]d op JazJ9,] op UONeUIIxoIdde sU_ *1919)UT Pp 
Q[QeURA ke] 19 UOTDZIEs Op sai[Iqeqoid so] aWUS UOTe[9II0N ap JoUVSqe | US saTJHSNf 919 & B[NULIOJ B19 “UOT[HUEYOI,| 
9p sptod sap sanejal oueueA ek] snid | red uoNnesgpuod e] e s[qenquye asepuos op uesd ap jaja ,[ JUSUIUIeINOD OUITISO UD 


aWnsaa 


pdaONAdS “d Aon 


W01}III9S IP S9pTIquqord say 39 
SIANSIUI SI] 9.1}U9 I[QIssod UOIL]I1109 9p S¥d Ud 
g[vsoul UoWeAQpuod oun Anod yuewtxoidde asepuos op uvjd ap jaya Uy 


eSt 


epeuey aenbiysiels 
GL-ESt “dd ‘Z .u ‘9Z "1OA 
000zZ eiqwasap ‘ajanbua,p sanbiuyoe | 


we "ap sit sei 
7 art pak as it's bi my, 


ie 
i Rig 
Cita? Pigae 
ot uy 1 y As tay no4 
2 
i 1 
, , 
eth gie ay 
j 
ia | 
th + 
i! ee ae 
re pees) LIAL 
A, ‘ <i f ee 
7 / 
iy wie 
y th 
i ta} 
eas 
. ‘hi \! 
y Li fy \ 
\ when ‘ w 
ra 
ho, Hh) 


Sper SNL FIN 
ST ounaee a Le { . 5) 
VAL aa aie air ee 
AN LL a ee BL 44 


be 


Ore Tie 11 ONL es sib 


' A e ae 


LP ce Seema Aah 

ee sf a“ 
a ey 

" i j 

a lw 1h r " 


th 7 yy enone ed i Oe We. 


iM fen is cy is 
ns er ee on 
ie atti if, iam. rf ane, oF ae 

b AR ew 
ren ee ATM Heer a ou 


erstary, ane 


Boar ae acai , 


i a « 
'y Wiehe) BA ulin oO Her 


abl ts Ma Ra rs 
, qreerinns Fal hgh? as eee 
aa oh, provhagies en By My Os Seep. Le anennctoa a a 

ites i. nite a See yi opaainise 4, al SG 5. 


| aoe uhm PARR ISA YM Ht al ars mi latest? alt 


ee He 3 7, oi ; Martens, aye “ Ae aA te “4 ras wey 
edi Saws OV o4 wv ey (Bb hi: may Piya an abs ' . tigcihh : 
-=4GOr wh Sam OT). Weert 3a \ } a Hit he ipiey 
Weettian! or Va im ein “ign i veh hy ‘and ay) 


; ’ ¥ 2465 oA? ote ale 8 Aine 
alt! ? wea tat ' : is i aati 
J 5 ’ 
’ ent iv ry {% 4 bay (htc! iL 
¥ , MP gk s ( 
i b : ‘ as J ; was 
ae Tot ued 
¥ i F sy + AF, oe 
4 ' i ; meee: Seip S 
fh ‘ aU FST 
eae Alees a 


$4 
x ; he coma 
i] we Ne 
"| a] bit y rrr ( ‘ny + * 4 "y “e ! 
er tice of Sat hie ate LW Ysa eink Se S fen i anergy” 
Ge Be ae OORT Re bee ae: _ yt 
ie ‘ Gil he i = 
P) tweet f 2 
a We TP oul t Mra 3° Qeeaeae 


L 5 ay 
aN bia _ hae san 


as i . 
OO tLga y* Ln 4 The; Lhe TaD Ny 
ne ee 
F| i . 
u te Ley 7 ha 
ie wiht, Foes el iy SLRs Seu, | TSCM A Tt ya 
aver hie Aine cela ieee f Pore hay, Gis Greta 
¥ ¥ ” z g ee it 
0 i <i Se ee ‘ Dh ada on 
yy ve ae a ee, ee val 08 | ES Se. iy creep ’ 
, ‘ ‘ fy a 
WHC i ee on ce nae BN. Co: ae ae 
mn ars agin + IP 


ry re Po Wars yasan" 
Atm et ren! ye Ve wn wabya' ea vende eonetar 

ieerrigne ae oe 
DAL aN W 


afi, A Ly 7 rahe" i 


Pun eat 


on fe’ ie papell 


u ' i 
ow aS Mesa na ie . i 5 | 
qaliaayy Pigunies ste i@ pw ie Bi TRE wemngge 


jlnbceol) oy are alps As e wet hi 4, 0 Lonoins: 

(WERAT RED Le” mae nia SBR: HEV Wap Aenbihle 
ag it Ho app? pa PHAN, 2. 
(Polke Pe emo cr aoe jee 
pester en, 4 i: ery 
4 aa werent 


‘0901 -6P01 “Hh ‘Sa1mawolg 
‘yoeoidde uoyenbs suneulnyse pozipesoues & :ejep [eUIPNyIsUO| 


Joj S[PPOW (8861) “S'd “LUAATV 19 “MN “ONVIT “TS “YADAZ 


‘OUT SININSUT SYS ‘ON ‘AleD ‘uonIps .p ‘9 
 UOISI9A ‘Z ‘aPIND S, 487) [V_LS/SVS (0661) ‘ONI ALA.LLLSNI SVS 


"Se]I9 A -JoBUUdS :yIOX MON “suljduvg Aaaing pajsissp Japo-W 
(7661) “H'f ‘NVALLOYM 19 “dd “NOSSNAUMS “d-'D “TVANUYVS 


“EST-Evl “07 
‘ajanbua, p sanbiuysay, “npuoyuod surstuedgu un uoyas asuodg1 
-uoU oun _p a}INs Jed yuaNONnb Jed saginduut sinayea sop ueusIdwi0S 
soguUOp ap seit) aJgnbue,p suoneuinse sop imod sieiq np 


SUOHIELOD (h661) 'A-'D “TVANUYS ?° “H “Aa “da “SLUNOONVA 


‘OST-LET ‘LL ‘uouDII0SSP JoosIDIG uvoI4aup ay) fo Joudnor 
‘sKoains o[dures ut asuodsasuou JOj sfOpoW (7861) WLU ‘ALLLIT 


“T9T-1ST “LL “uonDo0ssp [DINSIDIS 
uDILAaUL ay fo JDUAnOr “painduIl BUIaq a[qeuUeA Jy} UO spusdap 
asuodsai jo Ajt{Iqeqoid oy} usYyM sonjea Surssmu Jo uoneindwy 
(7861) ‘C' “ONVHOSGAIZ 1° “S'M “AORTA “Sf “SAF INATUO 


‘OSI-IS] ‘vpouny np anbysyvjs ajai20g ‘ajanbua,p 
Sapoyjau sap uoljdas Dv] aP 966[ landay ‘WIo\skS uONeWINS| 
poziesiousy oy} 10} uoNeyndut Jo souasesd oy} UI JOJeUINSO 
UOISSOIBOI POZI[esoues oy} JO soURLRA OY) SuNeUINS| (9661) 


‘dD “TIVGNUYS 19 “d “LYUNOONVY “H “AAT “A ‘NONDVD 


LSI 


"8E-1 “6€ ‘F a120g joousuwig jokoy ay) fo Jousnor 

“WYO |e-J_ Ou} PIA eyep dJ9[dwOoUT Wo POOYTOYL] UINUITXe | 
(LL61) “@U ‘NIANA  “W'N ‘CUIVT “dV ‘UaLSdWad 

"IOyYO [ae :yOX 

MON “Z9-L ‘(suaydaig “wy 19 OuNSOsy . ‘d'a ‘P_) ‘sanbruyay 
uf-fo-ssaupoon ‘siskjeue yestydeiy *(9861) ‘d'a ‘ONLLSODV.d 

‘(angered YW) ‘uoyvio0ssy 

JDINSIDIS uDvIidauUp ‘spoysay Yodvasay Aaaing uo uoljdag 

ayi fo s8u1paaz0ig ‘asuodso1uou s[qeioustuou jo a0uesald 
oy) Ul poyjew UOHeUINSS Isngol WY “(6661) ‘d-f “‘LNOWNVdad 

“C1T-LOT ‘opouny anbysyojgs ‘saxajduoo 

sajanbua saj 4nod ajpuipnjisuo] asdjoup ‘96 wnisodudcg :j1anday 

‘souUIWOSTe .p uosteseduroo oun ‘sonbrwojoyorp sopeurpnyisuoy] 

soguuop sep Inod suonnqmisip op uy ssuejouI uN,p songuiered 
sap uoneunsy (8661) “‘V ‘LLVNWAd }° “d-f “LNOWNVdd 


aIHdVaOOITsIa 


“Qpotpe 399 
ap auenb P| Jazorjawie ,p stwuad yuo mnb san soireyusWIWOS 
sing] Inod g1o0sse Inadepal J] 19 SAIIQIe xNap Say ‘UNO suey 
OU JOOIJOWIaI B Issne jUd} [J] “eae. 99 afqissod npusl 
e inb epeueg anbysyeis op soaAnesiumUpe jo so[euoIsal 
SOQUUOP SOP UOISIAIP kL] JOLOIOWIOI B JOT) Inne TJ 


XN 


SLINGNGALIDYaN aa 


000z a:quiacep ‘ayegnbua,p senbiuyoe | 


‘9)SNQOI UOeWINss Pp spoyjsut 
91199 asI[Mn uO,nbsio] 39 sogindwi soguuop ap souasgid 
Ud JOURLIVA L[ Op UONPUINSS,p so[dwIs sopoyjeul sap 
JOn[VAY ,P JULSSOIDIUI WeIOS [I ‘INNy [TeAvI] UN sue ‘aqui 
-ous! sed jso,u Inb ssuodal op suIsTURSOUI UN & OP steiq 9] 
alInpel op youLad inb ‘(Z'¢) aJapowl np sinaua sap syITeuLIOU 
op sseujodiy | & woddel sed a)snqol uonewNsa .p spoujour 
oun Jajuasgid e g}SISUOD B Z[DTUe 399 ap Jnoalqo.T 
‘onbuosjenb uorssaiz91 oun .p sangwiesed sop 
UONRUIT}s9 | Inod sagindui sinayeA se] No sprod saz JestyuN 
yelinod uo ‘ajduiexa Ieg “uOneWINsSa.p sadA} sanne .p ev 
sa[qeortdde stojayno} Jus UOT}Des BWIgLIeNb FY & 39 OUIAISION 
RI & Sagjuasald sapoyisU say ‘ayqesousi sed ysa,u asuoda 
op oWistuedgWI 9] onbsio] oUUSAOW 9UN.p UOTeUINSS,| Op 
owg[qoid ne juaWaNbrun spreye 1Sd,s UO ‘afore 399 sue 
“OUIQUI-d][9 { 
I[QULURA P| Op nal] ne (¢°¢) B[Qpour J] SUBP SIOLIBOIpUI SsoTq 
-BLIRA $99 IASI[IN 39 SOOLNVOIPUI SA[qVLICA ap JURAIOS OS UD 
A Qeuea ek] Jasuiosg}eo yUoWsTe39 jeLInod uO ‘se[gpour 
$90 suep SajUepuddapuUT Sa[qeLIeA souNe .p dInjour yWeLnod 
uo ‘a]duloxe Jeg “aug{qoid np ainjyeu vy] UOTas satjIxa;dui09 
ang juoreuinod (¢°¢) 19 (Z’€) Sa[apou say onb 13}0u ze 3s9 [J 
x I Inoyewiss | JUsWIgTNoWed snjd 
19 (Z'€) UONDIAS RI Op sINdJeUINSd Sa] asUOARy Inb ‘gyo1[duuts 
OP BIW 9] Ins sseq ag WeIAOp InoJeuNss uN.p x1oYyo 
3] ‘seo sao ap ane J no un; sueq °~” i rl owosIun esuodeal 
op ouistuecsut un asoddns inb inayeunse,, op uondas 
-X9_] & ‘SPSILIG JUDUI[QIeJ JUOS SINIJeUINSE So] SNO} ‘9AdTP9 
$2J} SO { 19 Y BUS UOTJL[PION ap ywoTdIJJo09 9] onbsJo] 
‘QUIQUI oq] ‘“asuOdgI-UOU ap xnF} IIQGIej Sat] UN,p sdUESgId 
Ud SgSTeIG Nod jUOS SINIJBUWIISA Say SsNo} “JUSUTWOPIAG 
“sgjqeuuostel qos (¢°¢) 12 
(Z'€) BJapour ne sagioosse saseyjodAy say sayno} anbsso] enb 
S9ST[NN IQ OUOP eIADP OU SPOISW DIVIWIOp 97}9D ‘sINeLS 
Sop gyljeuLou ap asgyjoddy,| &,nb (Z'¢) afgpoul np guprfea 
RI & a[QIsuas snjd a10dUd IdIBAP ISIS d][9 ‘AOUL[QUIOSILIA Op 
WINUITXPUL Np spoylau eT BULNd “(Z'¢) efapow ne yoddes 
Jed onb sinauie sap syyeuuiou op sseujoddy,| e wodde 
Jed Juejne a}snqol sa aporye 390 suep sasodoid uoNeUIss .p 
epoyjew ey onb yuayjUOW Inb uOne[NUIIs op opnig one 
aUN Pp s}el[Nsgi Say ajUasaid (666 ]) JUOWINeAg ‘snid oq 
‘gjqeiayoid yso 
IN] pore 499 suep agjuasaid a}snqol UONRUWINSE,p spoyysul 
eR] onb yuoTUOU UOTD—as oUTQINbUTID eB] B 9}LID9p UOTe[NUIIS 
ap opnig,| ap syeynsai sat ‘aatyusnf sed qsa,u asgujodAy 
BIgIUJep a190 anbsioj ‘anuos Ieg ‘agiidoidde ang inod 
OUR[QUIASIVIA OP UINUITXPU NP spOyIoW! B[ op UONesINN | 
‘agus so (7 €) SV9UI] UOISSOIZOI Op S[QPOU! Np sINeL9 
Sop gi[euoU ap aseuodAy,] anb ya aysIpeal Iso s[quioUsI 
-uou asuodai ap auistueogut un,p aseujodAy,| onbsoT 


NOISSNOSICG °9 


gjqeioubi-uou esuode-uou ep soueseid Ua UONeWIISE,P Bpoyjel Sus) -JUOLUNeEG 


"BSIDA-9OIA 39 99A9[9 OUUDAOU ONbNeIpenb 
INIL9 BUN OAR B BDULPUD) SO} OUN 4UO sasteIq sen JUS 
Inb sinayeutysa so onb onbseures uo ‘yayjo uq ‘ouusAoW 
anbuespenb inaxa | asATeue uo nbss0] sinofnoy juanbiydde .s 
JNe[ar sterq np osA[eur | op Aned e set) SUOISNTOUOD sa] 
‘a[qeordxo juowaTIoyjIp snjd sa inb 
90 ‘gsieiq juowgiqrey asar 74411 grgpuod inayewnsa,| ‘s1Oy 
-9INOJL, “SINaLe sap 91I[eWLIOU ap ssayj0dAY_| Ins sgseq sd 
SOULTQUIDSIVIA OP WINUIIXeU Np spoyjsul ey anbsind ajueu 
-oidins sed jso,U UONeIeISUOD 9}}9D “aINOT[IOU! eI JUSTAOP 
aISNQOl UONLUINsS,p spoyjou ev] onb uoNdsoxa,| & ssanq 
-LSIP JUSWaT[aTUDUOdxX=9 JUOS sINaIa saj anbs1oj sInofnoy 
juonbijdde .s yuopgogid oydessesed np suoisn[ouos sa] 
‘IOIWIOP 99 & ISeIULAR JOB9] UN JoUUOp 
SIOJOINO} IneJ [J “(7°Z) soayndui sinejea sap yueuarduio0s 
InoJeWINSd | 39 ([°Z) s1gpuod InoyeuII}sa J ajUS sadUaIEITIP 
op nod ‘fe19uas ud ‘a}SIxo [I ‘oQUUOp opoyjeuT uN INOog 
‘gsteiq snd 9] so 7 es Tl INdJeWINSA | ‘SINDJEWISS SIOIWIEp So 
Tuueg ‘e[qeiousi ssuodal op owistueseul UN 110) eB yuasoddns 
inb xnao onb aiqiey snd jeer stetq un yuo s{qQeIOUsSI 
-uou asuodai ap oulstuesgui un juesoddns tnb sinayeu 
-)89 SY] ‘(['p) UONdes BI & aILIOVp sIsnqol UONeUWINs,p 
dpoyuj9uU ev] Jed SIAINs ‘sognqUIsIp JUSWIOTCULIOU JUOS sINdLO 
Sa] onbsJo] sINIT[IOU! eI ISO DOURTQUIOSILIA Op WINWIXeUI 
np spoyjsu ey onb onbiput jnejor srerq np osAjeue J 


020 “00 Or F00 * 200” “ero pee fe 
EC SO0ln LU SC IT OW S00. cre ghia 
SEO > S00 PIF SE STOn aE COO 4 OS x14 
SO ISCO MEL MLO Oe wou eT OEM Syl 
970i: (US0'O SI'S nHO'0 &S 1Z0'0 450" Pan 
80 00 90°91 970 0 rs oa 
671 SOO 099% 980 p00  €8°LI pay 
SagNqUsIp JUaUIA|[aNUDUOdXe sINdLA sa] DeAe UOHE[Ndog 
6E0 > (Ou ESL & C0 Oneec€0'0) 7990 pores 
EVO 1200: ACO 1 F0l0.60'02» EO eG 


680 £00 PrLLl LZ0 ZOO 3 OS‘S 2601 
Is0 900 ZI‘OI 9800 £00 or 'I ee 
LLOm, OL Our eCT e500 wet: Onee PLO sy 
060 £00 ZO'8I 870 ZO s9‘¢ aD 
cel 00 89°97. 780 €0'0 06°91 aad 


ssonquysip WUSWOTeULIOU SINGS S39] SOAR uonejndog 
WWOgd la (%).ud .WOFTN La (%),ud rsierineer 
BOS = z¥ %08 =u 
rf QUUDAOUI B] Op SINDeUINSE / 
Joredulod ap jueyouLIed suoNne[Nulls sap sjeI[nsoy 


qT nesqey 


“jueusidins 
sed 3sa,u tnb 99 ‘agaai9 snjd 3s { 19_Y 9Ue UONe[9LOS P| 
onbsio] s]qrey snqd ysa sinayeUlNse so] sno} ep suusfow anb 
-lyeipenb Inaa,| 39 JUe[oI Stelq 9] ‘SINaLIa sep UONNQUASIpP 
ev] ouodut nad ‘anb anbrpur neajqri 99 op asATeue J “uon 
-R[NUIIS op epNnig,| op sieI[Nsgd se] sJUesgid | Nee[qe} oT 


OSt 


“(l= 1) + “s = ,.WOdY ‘ouuckour 
enbnespenb INdad,| Ip sad Suse RB] op UOTeWISS oUN 49 
‘000 I/* ‘sha /OOL) =.LA ‘JHeTeI sterq 99 B agrd0sse odAy 
Cee ap uonewnss oun ‘%oot x [N/(n- .n)] = ag 
‘(asejusoinod ua guilidxa) Juels sterq np uorreumnse oun 
g[noyeo & uO UoUTeULY “JUOWIOANIedsoI Suda tl sagjou 
‘SUONIBUINSS OOO [| $20 Op DOURLIVA PT Jo QUUSAOU PT ‘UOTIE] 
-ndod anbeyp inod ‘gjnoyeo ayinsua & UGC ‘sayuepgogid suoT 
-09S XNp xNe SajLINap SUUDAOUL PI 9p SUOTJEUINSS / SOT 9[NITVO 
® UO ‘sjuepuodal ap sUOT[HURYDE YOO I Sep UNdeYD INog 
‘(axl uaXouw asuodal ap xnej UN Inod) sjusWIsNe 
uonendod ap ay {18} e] onb ainsowl B JONUIWIIp JeIASP GOULLIA 
ey onbsind soueuea Ingj Ins onb 193n][d sInsyJeu}se sap sTeIq 9] 
Ins aseyduls [ NOUNS a9 op youLIAd (QO [) epueIS JUST 
-SATJe[AJ OI [Ie1 ap suone[ndod sap Jaigues ap ey 9] ‘snjd oq 
‘oseUuUOT[HULYOE, | & ONp Inde ounone v A UI .Nbstnd ssuodg1 
-UOU PJ & ONP INILI,] INS JUOUII[NES JaUIOUOD Os Op 93k} 
-UBAB,[ B199D “(O00 I = N = ¥) JUoUIasUdD—I UN TdT oNqDOIJO 
uo nb Ja}0U & S9 [JT “8S6‘0- 19 SRI‘ [- aU OLA oTNQUIvIed 
9D °%OL ep ios uafour asuodas op xnv} a] anb ayos 9]]9} 
op suoneindod p sap sunoeyp Inod juourgredgs guIULIa}9p 1s9 
°n angureredsy ‘¢‘Q = "n soak (¢'¢) a[Qpoure] UOJas sjuepuodgL 
Op SUOTTHULYD9 OOO I e[NuIs v uO ‘uoNetndod onbeys nog 
‘(40S = -Y) BOS Op ured ne uoreyaL09 ap JUSIOIJJ909 uN e 
juepuodsaioo ¢ & [B89 adAj-yIeO9 | B ONE, 19 (%08 = 7Y) 
%WOY op sued ne ( { 19 _Y BUS) UON[AIIOD op JSIIJJO09 
un B yuepuodsaLio; ¢‘|{ & [e389 0 adA}j-Ie99,| Be UONe[ndod 
aun ‘sUONNGLISIp $90 op suNdeYD INog “(9 - (9) dxq ~'3)0 
RB sonusde1 © dsuUdsXoW ap ajeyuouodxe Io] e] 
JUBAINS SINALE SINI JUO sane xnap S2j 19 ((,9°0) JON ~ 3) 
Q[euLIoU IO] kB] JURAINS sine sine, yuo suoNelndod 
xNaq *,0 SOUBLIBA ap Jo 9[[NU sUUSAOU Op 1UOS 39 'Y Sol 
JOA Sd9[91I09 sed JUOS OU ‘saT[a aIJUa SoyURpUddepuUT jUOS 
'3 sInalia sx] “| suUsXOW ap a[[aNUUOdx~ Io] eT JUSAINS 
32 So]Ja aUe sojuepuadapul yuos ‘y solOyegTe saTqeLeA 
“say ‘¢€='g 10 7 =°Y done (Z'¢) B[QpoUr 9] UOTIS COO I 
a[te1 ep suoneindod p gjnulls e UDG “UOTe[NUIIs op opnie 
oun 9nVdejJJo B UO ‘sajUspgogId sUOTIDS XNaP xNv sg]UIS 
-91d sUUdAOUI P| Op SINa}JeWINSe soy Joredul0d op ulyYy 


NOILVTOWIS Ad aAGdALA °s 


‘(€'p) uoneuntxoidde | aonb swigui ap setidoidde yuos (¢"¢) 
10 (Z'€) SPTQPOW sa] Is SAQUIIISA UdIq ag JUsTeIAp asuode1 
ap sai[iqeqoid soy ‘apote 390 suep egsodoid uoneunse .p 
QPpoyISU ek] op Uas as UO.Nb Ja a[qeJOUsI-UOU 3so BsUOd9I 
-UOU BI oNbsIOT “SAgUINSS JUIUID}DALION JUOS asUOdaI op Sat] 
-Iqeqoid so] onbsi0] seo 9] 189 Inb 90 “('a) “7 yUDUIO}D0LI09 
aunse ('a) “ wt onb uONIpuod & apIyeA vias apoyigut 2119) 

‘ya! inod ‘'a snpisgi sap uonouoy us [('2) nay _® 9p 
onbrydess 9] Juestey Ud SINaLa Sap sITTeULIOU ap ‘Sssemodcy, I 
JaIJU9A jnod UO ‘IsUTW “sonUUOD jUOS UONDeTIS Op S9IITIq 
-eqoid sa] ajjanbe] suep ueunas A, 39 UOSSUaMS ‘TepuRS ap 
Q[NULIOJ LT B JUSWIIITEIJUOD SA9UII}Se JUOS asUOdaI ap SEITTIG 
-eqold sj ‘uonenbs asgrusep 9109 suep ‘anb Ja}0u & 189 [J 


61 


Ci), ai 


. ! u 
(4). ayy 55/9 49 le = ( a) A 
(661 “d ‘7661 UeUNOI 
yo UOSSUAMS ‘JepuIeS) Jed sQuUOp ag ynod sgUINse onb 
-utdwio uontyedes ap uonouo] eT ‘soyesout soyiqeqoid sone 
juapuodal uo][HueYo9 | op sayun say onbsing ‘sayuepuodgi 
sgiiun sop uokow ne onbiidwe uonsmedgi ap uonouoj ev] 
JOWINSS JOP UO sIew yUopgoeid oydeisesed ne nb onbruyse} 
QUIQUI B] Jast{4Nn ynad uo ‘asuodgs-uoU ep souasaid uq 
“QUISLIO | Jed juessed 
9/]| ajued op ailosp oun ,.p 3uo] 9] JousITe,s said ned e uaTeIAp 
enbiydeiz 99 ap syulod say ‘sognquysIp JuoWSTeWIOU 
UOS SINE say onbsioT ‘onburduwe uontedel op uonsuoj 
RB] 1S9 (*) “4 39 pIepUeyS STeULIOU IO] eT JURAINS aIIOJegTe 
J[QeUeA UN ,p UONIWEdaI op UOTIOUOJ BI SO (*) NO ‘G3! 
inod ‘'a snpisg sap uonouoy ue [('2)"7] ,@ ap onbryders 
9] OIIeJ B BISTSUOD ([ [°7 UOHeNbe ‘¢z “d ‘og6] OUNSOSY .q) 
onbisse[o apoyjsu oun ‘asuodgi-uou op sed e A,u [I .nbsioT 
‘(sagarosqo sed yuos ou SInaLIa say onbsind snprsei sop 
gil[euLIou ap sseyjodAy J 19yn[d No) sinaLe sap syTeWIOU 
ap asaui0dKy ,| JOIJU9A & 9SSo19IUI 9139 nod UO ‘adULT QUIS 
-IGJA Op WINWIXeUI NP spoyIaU P| JOsI[NN.p ing a] sueq 


SAINIIAII 
Sap 9yjews0U ap asgyujodAy | ap UOHKILIIA = 7 "p 


‘g[oTe 399 suep saIZgTIALId 939 & INb spo} PT 1S9,9 Is 
surgut ‘' 39 % JoWNsSe nod souR[qUIasIeJA op UINWITXeUL 
NP spoysWI LT & SINODAI IIOAL,P aIlOJeZI[Go UOWINTOSQe 
sed jso,u [I,nb aUUOTUSUT STOJa}NO} yNeJ I] *(¢) odeia.] & 
onbnsizo0] UoIssei391 ap sInpgoold oun Jasi{yN snyd jnod ou 
uo ya onuuos jsa (‘x | ‘4 ) f anb guuop jue9 (¢"¢) afapoul np 
soneguiesed so] JOUII}Se Inod sour,quIssIeJA Op WINUITXeU Np 
Spoyjoul P| JosITNN e snjd JUSIADI OU BIV9 ‘UepUadad ‘aoTIe 
320 suep ogsodoid apoyjew ve] op usAour ne snuaj}qo xnod 
B SInatigdns JUDWIIIEB9] S}eI[NSeI sop oUUOP UONRIIJIPOW 
91390 onb juaUOW UON[NUIIS op opNj9 oUN,p s}eI[NSEI SIT 
‘(666[) JUOUINesg suep sgsodoid 919 & UOTIDES 9309 sUep 
giuasgid owIYy}WO3Z[e | ap (¢) odej9,| op UONVoTJIpow sup 

‘(€'p) uonewrxoldde | op oyyenb 
P] op 39 (€°€) 19 (Z'E) SATQPOU sop sUpITeA eT op oned 
opuvis us vipusdsp suusKOU! e] Op SINa}eUIIse xnap sed 
op risa ey “2044 Mt gu exes inayeUINsa 19] “O35! inod 
"NX 'g- og - Ye ‘y>a1 Inod ‘ ech ®g-'x='2 No 
‘sagjoadsau quatos ¢ = ‘y" ‘aS’? 19 O= bait SOJUTLUOD Sd] 
onb 39 2? S°"’? gsTuutUTUI UO nb 9110S sa ap ‘4 sogindut 
sing[ea” SO] JURUTWLIS}9p ud (¢: €) UONDdeS FT B SUID nuo}qo 
yuaWIae39 IS9 (Z'Z) anoyeunss,T °“2%4 41 gh0u vIOs Ind} 
-PUITISO 39D “aQUIT]Sa asuOdgI op suytqeqord episa (4), d 
no “(‘x),d/] sed ' wm juedSejduis1 ua nuajqo so ([°Z) 
Inayewnss JT (ZZ) No ({°Z) suUsAOU eT Op INayeUIT}Sd | 
Jistoyd ynod uo ‘sgummsa yuos (¢°¢) 30 (Z'E) SaTepoul sap 
sanguieied sayz onb stoj oun ‘(¢°¢) UOOeS eT B BUIWIOD 


000Z e:quiacep ‘ayenbus,p senbiuyoo |, 


“UONNIOS P| JOANON Inod 919eOTT JO] 
ye uoNNdsx9 ,.p sduia} 9] aNUS stwoIduIOD Uog UN WUINOJ 
SOUIYILIOSTe XNep s9d asp UOsIeUIquIOD eB, anb juoWONb 
-Uidurs juaNUOU sq] ‘uosydey-uo\MaN adAq ap opides snjd 
SWIYILOSTS UN & SI[eNIUl SIN[eA Sop ITUINOJ ayINsus Inod 
Wd 2d} op owyu0sye un eae JeIOI snssao0id 9f J20UDuI 
-UWI09 & 91SISUOD INb oITepMuUIs sydodde oun quasiynn (9661) 
eUuUlaq jo JWOUNeIg ‘9}X9]U09 one UN sueq “UuosYydey 
-UOIMON SWIYIWOZTL,| Wp VoUIsIOAUOD eT JoINsse nod 
SOUUO JUSUTLULSTJJNS SO[eIIUI SINDTVA Sop ITUINO] ossind [I 
‘UAWZ[Nes suOHeIgi sonbyenb seide nb ay0s a][a) ap a1gags 
nod sdUsSIOAUOD ap a1Q}LI9 uN dade gsodold sUIYILIOZ Te | 
JasI[nn .p wesyyjns [J] ‘uosydey-uoimon ow iwo0sye | aonb 
[91 opides snjd ouyjWOs]e un & SoyelyIUI sINaTeA Sap ITWINOJ 
Inod gn san Jargavs wWeunod gsodoid suyiL03ye J 

*Q0UdBIOA 
-U09 op sourg[qoid sap ganoldg e uosydey-uoJMON SUIYILI 
-03[e | Jenbay Inod seo sanbjanb say inod yney-1o swiyyt 
-O3[e | & SINODAI IIOAR NP & UO “UepUsdad “JaBIOAUOD kB 
opidei snd yso nb uosydey-uoman SUIYILOSTe | Inod 93do 
19In[d & UO ‘UOTIDas oUTeYDOId R] op SUOTe[NUIIS soy SURG 
‘90ur]q 
-WSSIVIA Op UOTOUOJ oUN sed astwIxeU OU UO 1OIr,nb UOT 
-daox9,J 8 ‘(LL61) UIqny 12 pure’ ‘Joysduraq ep WY ew 
-OB]@ | IAB SIOULTQUIASSAI SoUIeIIOD opessod sUIyILIOS]L 
399 onb Jaj0U B JUDWII]eSE Js9 [J “9SI[IIN 919 B [I NO Sed Soy 
SNO} SUP JBIOAUOD FB [I ‘WeJ UG “IOSIOAUOD Op ULAR SUO!} 
-e19}I .p dnoonvaq sipuaid assind [I ‘seo sureyiao suep ‘aonb 
-onb uonnyos ey J9AN0y) Inod soevdIJJa Sat} jSo OUIYINIOSe 
300 ‘onbyeid uq ‘sagiisep suoneUInse soy Jtua}qo Inod aonb 
-IISIZO] UOISSOIZ9I1 op sINpgd0Id ouUN_p Jo aTIeQUI| UOISSoIBOI 
op ainpgdoid oun,p Josodsip op yuowionbrun suop yWyJNs [J 


“¢ odvy9 | & IOUINOJII 19 [ + 
f= [1asod uous ‘ayuIa}Ve JS9 DDUIBIOAUOD FJ IS Ja}QUe “9 
‘(Ad 
Jtuajqo Inod onbnsiZo] uorsseisei op sinpgooid 
oun juesI[N us (Zp) suotenbs .p auiaisAs a] SIpNOsel *¢ 


£9.31 Inod 
Oe js + ( nd = i A Jed soyuenbuew singjea soy soynduit “py 


: wo i) a8 J1U9}qg0 Isute Inod agigpuod 
UOISSOIZ91 Op sinp9o0id aun juestyyN us ‘(,_p( qd 
‘gsuodg1 op saijiqeqoid sap sayueinod suorneulnse 
sop usXoul ne ([‘p) suonenbs.p owiaisAs a] sIpnossl “¢ 


‘suolyes9iI,p siquiou 2] anbipur /no ‘|, = /sasod 


‘sojuepuodg1 sora Sg] $ayno} Inod | = @( '{)d sasod 
g[duroxo sed ‘( p 19 °n sanguiered saj snod no) asuodg1 
Op sgiiqeqoid say] Inod safeniul sinsfea Sop JOUTWID}9p “| 
JUBAINS 
OUIYIIOSe | op uakow ne nuajgo ang ynad (Zp) 19 (Tp) 
suonenb9g,p saulaisAs Say JUSUIDANBUIO}[e aIpNosel onb Js9 
(¢‘p) Uoneurtxoidde | op ajuessaigjuI 9}9UdoId anne su: 
“QOURTQUIDSIVIA Op WINUITXeUl 
Np spoyjswu ey JastNN.p sjqeigjoid ong ossind [I ‘seo 
90 suep ‘anbionb (¢-p) anb suonewrxoidde soino{[Iowl op 


JOANOI ap VIqIssod jsa [I ‘agUINSa UaTq ag ned no snuUOD 
1S SINSLE Sap UOTINGSIP eI Is ‘npuajUS UdIg ‘sINoLE 
Sop UONNQI}sIp ey Jotyiogds op sed yatnbas ou aya nbstnd 
SINeLIa sep gilyeuoU op ssaujodAy,, e@ wodde1 sed 
dISNQOI alg WeIASp aIJ9 ‘suds 90 Uq “‘Y kB a[JaUUONNIPUOD 
‘X 9p uoNNqsip e] ap yuoWIoW! Joruaid 9] anb JoBIxo.U 
ap osejurAr | spessod uoneurxoidde ayag a[nu ysa aus 
P| Op OULIA} SUIgIxnep np sdueJedsa_| anb Ja}0uU ¥& 389 [] 


(Er) -(x'd + d)d= (xa) d= (x Cada 
‘OLS 9}}99 Op SOULJA} SJomWeId xnap sap souvi9dsa | 
jueuaid ua yo (‘v|‘4)q yutod ne sanyeag JOA], op atas 
oun .p uakoui ne (‘4 )d yueste9uly ua snuaigo ayuRArns UOT 
-ewrxoidde | inod 9)do y9yn[d v uo ‘apore yo0 sue “aJAINS 
vA Inb 99 suep ogigpiIsuod sed elas ou 39 onbryeuojUr 
ana 9p julod np yueyodut uonnsgxe,p sdwia, un ywaornbas 
spoyjsul oy00 ‘“Uepusdsd ‘asuoda1 ap siiqeqoid Inez 
OP ISISAUL,| B JUSW]TOUUOTIOdOId soyURepuOdal saqIUN soy 
JURSSISIOYS Ud des}oog Np apoy}eu PT INs saseq s1391e.1S 
oun Joddojeasp ap sjqissod juawayess yesas TT *(S'¢) 
uonewtxoidde | uOp ‘soasestauo ong jUaAned sinarsnid 
39 uoTeuIxo1dde oun JasI[yN Wop uo ‘auretqoid 90 eB dORJ 
oNej INOg “sNuUUODUT JUOUT|QeTQUIASTeIA 3Sa Inb ‘'3 sinaLS 
SOP UONNGMISIP e] oJeUUOD ap sIOJajno} YWoInbal soueI 
-9dsa algtusep aneD *(Z'p) uonenbg,| suep (‘x | ('4) 4) q 
Jg[NIed op sINsoWI US 9NQ INeJ [I ‘Vl9D INOg ‘ajUIOIe 
WHOS BDUdBIDAUOD P| onb 99 v nbsnl (Zp) 19 ({"p) UoNeNbs,p 
SOUIQISAS SO] JUDUISATILUIOIE DIPNOSal & d}SISUOD UOTINIOS FB] 
JOANON op Jounad inb auryjwOsTe UA, (Zp) 19 (Tp) SeasteIq 
uoU UOTBUINSS pp suoNeNbs soy JURA[OSaI Ud SaNnuUd}qgO jUOS 
" 10 ° *'g <°g sanguiesed sap suoneumnsa soy ‘IsuTy 

‘Ll =41290 = 4 1nod 


"DC os 
(Gohl en Ne A010): tal ra BAKE 


an 
(a) a} - anr¢ 


suonenbe.p ae 3] SIpNoses 
19 ‘a[duroxe red ‘oour[quissiesa Op WINWIIXeW Np spoyssUl | 
red (¢'¢) aJ2pour np 'n 19 °n saneuresed soy JowNse yeunod 
UO sJoye ‘(¢'¢ B[QpoW Np soneuIeIed sop s1pusdgp sed yop 
OU STeUL o[eULIOU JUSWaITessaogu sed) anuuos yea (‘V1 A) 
guyiqeqoid ap sysuop op uoOUOJ eT Is ‘oUIQUI 3d 
‘gsuodal ap suIsIuvI9U 
np uou 19 UOT[HUeYDI,,. Op UONde[es ep sUISIUROgUI 
np uonsanb 3sa [t,nb uondeoxe,| & ‘olepMuis JuowNsse 
un inod (6[¢ ‘d ‘7661) UeWNIAIAA 19 UOSSUBMS ‘[epueS 
@ QAOAUAI 1S9 IND}DII FY “a[QPOU! np supreA ey sWodut 
nod (asuodgs-uou op no sed year A.U IIs SnuSity cig 
uo, anb 'g 10 °g ap sinayeuInse say jUOS inb) 'g 10 °¢ 
Inod asuodgi ap austuvogur ne Woddes Jed s}ue3J9AU09 US 
({'p) uonenbg,| ap ueXow ne snuaigo 'g 39 Og sinajeul 
-Ns9 say ‘JaJJo UT “(Z'€) B[Qpow! nv yoddei sed sssaysnqol 
gureyiso oun youleye39 oinsoid asuodgi ap siufiqeqoud ey] 


@iqesoubi-uou esuodeJ-uou ep eouesaid Ua UONeWIISE,P epoUjEW BUF) -JUOLUNeEG Srl 


Op ISIOAUI | Jed JoIgpuod op tej VT “sundae sap UONGLNSIp 
eR] Jotpioads op sed a81xa,U Salvo SdIPUIOW Sop BpoyoW 
ey onbsind ay1jeu0u ap aseyjoddy,| @ woddes sed asnqgol 
agigptsuos yso oyooidde aneg ‘y= “7 10 1 = "7 no 


qd 

Cp) Wks Ole 7x!) -'g) a ak 
suonenbe ,p auiaisXs 
J] SIpNosal B aUsTeAINbe yso UONESTUNIUT aNEDQ *(‘4)d 
asuodal ap ayttiqeqoid e] op asioaut | red sagsgpuod gud ne 
SINILID SOP SUMWOS P] JOSITUITUIU B WeIAISISUOD (Z7'¢ B[QPOW 
ne yioddei sed onb sinaue sap siyewiiou ap esayjoddy_| & 
yodde sed juejyne) a}sngol UONeUITSS Pp spoyIsU oUN sIOTe 
“‘UOT[URYDE | 9p SeyUN so] sojnoj Inod o19z B saInetigdns 

ya sanuuoo juareyg (‘4 )d asuodgs ap sait{iqeqoid say ts 
“aisnqol 
snjd uoneUss.p aspoyjeul oun eB ILINODEI op s[qQeiojoid 
4S9 [I ‘9TOIA JSa asgujOdAy 9}199 anbsioT ‘aoueLeA SUIQUI 
RP] UO 19 SOINQLISIP JUSWIOTeULIOU JUOS SINELIa say puenb 

OPl[VA ISO DOURTQUIDSIVIA OP WINUWIXPW Np spoyjoW eT 


A 9p pusdgp inb ssuoda. 
ap euIstURIIW UN AnOd UOHLUISE,p ePOyVIIA, Tf 


‘(Z'€) 9[Qpow np snes 
Sop su]TeWOU ap aseuJOdAY | JoIJIIQA op juRyoULEd (Zp 
uojoes) onbrydeis apoyjaw oun nb auigu ap ([p UONdes) 
g[qeiousiI-uou asuods1 ep owistueogui un nod segsod 
-O1d UORWIISd Pp BpoyIsU LP] DILIDEp & Las UOTIOES 39D) 


aaSOdOuUd NOILVWILSA.d dAOHLAN ‘“F 


‘(Z'€) eTQpow np 
Ople | & B[qeUeA 93199 asI[apouwl pea UO Nb Ja UOT{HUeYDE, | 
ap sauN sa] sayno} inod x s[qeLUeA ek] 9AJOSqo WeAe UO 
* Is sagjoadsal 939 JUSTeINe soJUapgoeid sojUIeNUOD xNep So] 
onb jsa aysoidde ay1909 algiuap onbiso] ey] “¢ uOT{NURYDE | 
op oyftey ep sau no ‘us'y S144 Of = 4H :omsog,s 
sloje jned ouuekow ve] ep INayeuINse,J “jUsWeAnoedsar 
'g 19 °g op suoneumnse say uos 'g 39 f yo ‘GQ 32 snod 
X Id - Od - 4 = ‘a ‘yas aod ‘ly Iq - Og - ‘a = ‘2 no 
‘sagjoedsal quatos Q = 'y'aS?"{ 1a Q = 'a5?"¢ saqurenuoo 
sat aonb 49 wane astuut uo.Nb sos aI[o1 op ‘x 
soogindull SINZ[eA SII JURUIWIA}Ep US NUd}qQO ag Jned (7'Z) 
insyeunse,J “441 gjou eles inayeulNse 9D “aguINsa 
asuodgi ap siiqeqosd ej so ('4), 7 no “('4), d/] sed "nm 
juede[dures ua nuaygo 1so ([°Z) InayeumNsa,T (Z°Z) no ({'Z) 
QUUSAOUL P] OP INIPUII}SI,]| IIstoYyd nod uo ‘sguINsa 1UOS 
(€°€) 19 (Z'€) SoTepowl sop sanouresed say aonb s1oj ous, 
‘(0661 
OU] SIMINSUT SVS) SVS [919180] NP NIIN eNpgoo.d | ap 19 
uosydey-uoIMON SUYIIOZ]e | Op opre | & (p'¢) UoNeNb9 | 
OP UOTLSIWIXeU LP] SNJOoJJo & UO JUSWOTVULY “UONe[NUIIS op 
apnig oun sIpusidanua ,p JULAL SIOPISUOD JUSWASNALI9S ong 
jueASp o8ejURAR UN 3sa INb 99 ‘anbyuewoyuI ana op jutod 
np opides snd 3a ajduuts snjd 3sa afjo,nb sored onbursurnu 


Lvl 


UOTILIZIIUI,.p spoyjow oun ev uoNeUTxoIdde a}199 siajoid & 
UO ‘d}ULISUOD SOURLIVA DUN JUO Jd SdQNQUISIp JUSWISTeULIOU 
jUOS sInaua sat onb sseyjodAy,] ins sgseq 3so UOTyeW 


-Ixoidde oneD “uct /E fol = 2 10 [+ MY 2 9fi1 =¥ no 


icy. np "n+ %n] = Wehiey ae ; : 
ce cen 


(2861 Healy 0 suery ‘1o807) a1UvAIns UONeUTXxoIdde | 
Inod gido 19ynI[d & UO ‘afore 399 sue (786) BSURYOSIIZ, 
39 D090y ‘Sdo[US0IH suep swWIUIOD onbisuINU UOTeI39}UI /p 
Qpoyjeul oun juesi[NNn us Joate Aynod ug ‘(‘x | (‘4)d)q 
Jowrxoldde .p oinsow ud ag Wop uo ‘UONeSsMUIxeU P| 
Janjoojjo Inog ‘7 JUestuTxeW Inb saneurered sap sinajeA Sey 
JOANOI B O}SISUOD BOULTQUIDSIVIA OP UINUWIXeU NP spoyjsu 
eT +, sournea ap 19 ‘y 'd + °f ouuekow! op syeuLIOU Io] 
aUN,P su[Iqeqoad op susuap ap UoNoUO} e389 ('X| 1) f NO 


(S'€) « 


(ve) ‘LCxiCa a a- ul O"™ + 
(xia sCadyuy P< =1 


11L199 319 
jnod 7 SOUL[QUIASIVIA SP UOTDUO} L] Sp JoINJeU SWUWWeSO] FT] 
Salo arjUa sojuepUodepuT JUAIOS Jo ,O a}URISUOD BOURLILA 
oun juare ‘(Jask[eue eB saguUOp ap ad} 2] JInod aju0U 
-IWod UOTNGUIsSIp ane 9]N0} NO) ssenqLsIp JUOWATeWLIOU 
quatos ‘3 siInaia sat anb oireyuoutgtddns ssamjodAy,| yuep 
-uadao yarnbal apoyjsul 9319) ‘UIJ 9109 B SasT[NN ang jnod 
QOUR[QUIOSIVIA Op WINWIXeW Np spoyeW eT “(¢°¢) 10 (7'€) 
SO]QPOU sap saNgUIVIed So] JUSUIQURI[NUIIS IOUINSS & 9}SIS 
-u09 ayooidde aina]jioui oun ‘uoneNys a]]9} oun sue “s[qQReI 
-ousl sed 3sa,u asuodgs ap oustuesgu 3] onbsJO] sopryea 
sed ouop jos ou (Z'¢) UOT}9es ev] ap soydosdde xnap sa] 


(AIL ='wd = (4)d no 


CXC Da - 1 


! ! 1 -'y'¢+ = (x 0='NI'A) a 
CX1CA)d° 4) 09 


onb JajNuoUL sInaTre .p 
jnod US “}2I9IUL,P I[QULIRA L] OAV JUSUIOANLSOU I9[9LI09 
iso osuodgi op ayiqeqoid ey] Is wey ong ynod ames 
JUSWIOUUOSIE! UF) “SaTUNdI s9}IUN sa] soyno} op suUdAOW 
Rl & SINILIQJUI DUOP vas Jd saJUBpUOdaI sg}IUN Sop aI[99 
& INILIgJUul elas soyUepuodsI-uOU sgyIUN Sop oUUdAOUI Pe] 
‘aguuOop Y op Inayea oun Inod ‘sJo[e { 1QI9IUI ,P I[QRLIeA Pe] 
DOA JUDUIDATISOd dg[VLIOO Jso asuOdaI op syT{IQeqold ey] Is 
‘gduroxe Jeg ‘(7861 BuLYosaIZ 19 9Dd0y ‘saoTUDAIN) ‘y'g 

+ g# (xX 0='yI'4) a anbsind (z'¢) uonoes eT B aILIO9p 
uodej ve] op sayuenbuew sinajea say Joyndut sed jreunod 
QU UO ‘SaJUdBIBAUOS SUOTJRUINSE p yWesodsIp UO Is DUIQ\I 
‘soyjnu sed quos ou ({='y1'x'3)q 99 (1='yI'3)q onb 
gored sajuasIaAuod sed yuOIJes ou saljQuIeIed sap suoneUl 
-1]89 Sa] “Uepusded “(Z°¢) a[apour 9] JasI[NN Issne ynad uO 
“Q0UR[QUIDSTRIA Op LINWIXeU Np spore eT Jed 'n 19 % 


000z aiqwisacep ‘ajenbue,p senbiuyoe |, 


Inod UoHeUNss ouN 11U9}q0 .p s[qissodunt jsa [1 ‘sajuepuodgi 
sguun sat nod aonb agarasqo jso,U { s[quiueAa ev] onbsing 


[Cx "9 + %)- Jdxa + | 
I 


(e"€) Bri = wd 


:QJUBAINS UOSeJ eB] op osuodal op s1[Iqeqoid Fe] Jastfapoul 
nod uo ‘asuodgl op ousTURdgUI [9] UN INOg *(a[qQuIOUsI 
-uOU asuodgI op sUUSTUBd9U) { op pusdep asuode: op ourstu 
-RO9U 9] oNbsJO] Sestelq JUSWOLIOJ ong JUSANod jURUD\UTeUI 
e.nbsnf sayuasgid suusXoul ke] ap sinayewinsa say sno], 


A ap pusdap inb ssuodau ap suistuvsayy «¢"¢ 


“i I Op [No[ed 9] JoTJI[duIs op asequeAr | 
jusUIaTe39 sUOdWIOS 199g ‘sagyndult sIne[eA xne snpIsel 
op Joynole sed ou ap Istoyo & UO ‘doUeLIRA P| Op UONeUINS | 
Ins sed oylod ou apore yuasgid np yalqo | sulWI0D “snpises 
op ginofe sed year _.u uO | Is aonb sastogid suroul sUUaAOUL PB] 
op SUONPUIT}SA sap alinpoid B aoURpUd} ¥ aTTa ‘snjd aq ‘uONn 
-eynduil | & oNp soueLeA vy] JOWNsa,p sed sinofno} outed 
ou onbruyoe) aya0 uepusday *(966| [epusesS 19 UNooUeY 
‘saJ] ‘uousey ‘ajduroxa sed ‘NOA) oseUUOT[HUeYDI,| 
B@ ONP VOULLILA PL] JOWIISA XNaTW op Inq 2] suep seajndut 
sIngeA xne Npisel un Joynole yUaWalesa yWeLNod uD 

“§ UOT[HULYDE,| Sp SoqUN say sajyno} Inod sapryea sasoddns 
JUOS JUSWINDOP 99 sULP SaIEPISUOD sSaTepow saz sno} 
anb sJajou & 1s9 [TT ** 11 9}0U BIS UOSeJ 39390 ap NUD}qO Ind} 
-PUINSd J “S' UOT[HUeYDE | ap SoMUN Sat $9]N0} SAIASQO PRAT 
UO IS (SaLIVD SOIPUIOUI Sap pois eT Jed) sanuayqo jreine 
uo | anb suoneumsea say juos 'g 19 °g ‘Sey uq °'g 10 Og ap 
SUONLUINSS Sap SOUIQUI-saT[a JUOS Inb 'g 19 °g ap (sajuep 
-uodgi soyiun sap uadXoul ne seiied saIpUulowl fa soda 
Ry sed oe suoneunss sap juos 'g 19 °¢ no ‘ va 'd 

+ °g= 'X sed saguuop juos sogindur sanayeA Sa 
So] JOR 9[91109 UOU aI[NU sUUdAOUI 9p dITOJeg|e INI ,p 
dULIO] UN 3s9 ’3 12 snUUODUT sanguIvIed sap quos '¢ 19 °F no 
(7'€) cramer yd ood =. 11 

JUBAINS 

d[Qpoul Np jURAIas as Ua saJUeNbuULUT sINaeA soy JoyndWII 
qjnoed uo ‘(7'Z) Inayeunjsa | Jast[uNn yOynd asgzord uO IS 

‘X'd1j g0u BJ9s 

UOdSeJ 91199 op nuaiqo sUUdAOU BI Sp INayeWINsa,J “UOUTS 

0 10 puodal 2 gy1un | Is [| afesa inb onbrwojoyoip sfqeiea 

gun 3s ‘y 19 (a{duroxe sed ‘aouer[quiasivsA op WINWITxeU np 

spoursul e] ed) Jounse ¥ saneuered sap quos 'n 39 % no 


[xp + %)- Jdxa + | 


=(XI1='Y)d 


:aILIN9 S$ ned sfepoul 99 ‘UOT 
-UBY99,| Bp SayUN say sojno} Inod agaresqo CY) a[qeLeVa 
g[nes oun p anb asodsip au uo 1g ‘asuodgas op suiqeqoid 


e[ Inod stnbas ouop 3so sJepowl UQ, ‘anbnsiSoy uorssorZ91 
eun.p usXoul ne sguinsa ssuodes op siiqeqoid vy] sed 
/ be M/{ JUeNINSgns Us nus}go ang ned ({°Z) InajeuINsa J 

‘rl ouudAour eB] op uoNeUI 
-1]89,] Inod orrejuowgiddns uonjeunojut sumo squire 
91199 Jasin .p s[qeigjoid ssoye yso |] ‘asteiq JUOUIAVI0J 
ang ined ([°¢) INaJeuINSa, | ‘({ DEeAe dgTaLI0D IS9 Inb) y 
a[qeue ve] op pusdep esuodai op ouistueseut a] anbs107J 


X 9p puadap inb ssuodaa op suistuvaaq, 7° 


“syuepuodal-uoU Sa] 
sno} & sjuBpuodgi sap suusKoUL P| JURINdUIT Us 39 (7'Z) UON 
-enbg | JuesI[Nn us nus}go WeIne UO Nb Infao anb augUI 9] 
4S9 INDJBUIT}Se 192) ‘syUepuodal ap [eI0} aiquiou 9] 1s9 Yu no 


wu Ndy 
yal a * 
ul 
:SJO[® JUSTASpP (12) INBJCUNSs "TJ ‘gsuodar 
op duiqeqoid SUIQU BP] JUO UOT[NUeYS9 .[ op SeyUN So] SayNno} 
“suOjIUN asuodal op oWIsIuUe DOW UN .p asouioddy,] snosg 


Ciig) : 


dULIOFIUN ssuodaI ap sUISIUBIIFA, «= ['°E 


‘gjqeiousi sed 4sa_.U (¢°¢) UOTDeS P| 
ap Injao aonb ssoye sayqesousi yuos (7'¢) 19 (['¢) SUOTIOaS Sap 
asuodal ap sowistueseul say] “4 op puedap nb asuodal ap 
oUIstuBdeW UN asoddns uO “(¢°¢) UONDeS EB] B 19_Y op puadap 
Inb asuodai ap ouistueseul un asoddns uo ‘(7'¢) uoTDas 
RI & ‘auOsJIUN asuodal op suIsTURSoUI UN asoddns UO “({"¢) 
uoldes eB] Y ‘soauidoidde uoneumnsa .p sapoyjeuw sop a1UDep 
19 SAQUUOP sap UONaSIp e] 39 esuUOde ap suIsTURDOUL 
a] JUeUIZOUOD sasaujodAY sajudIaJJIP snos (ZZ) 3° (1'Z) 
SUOISSOIdx9 Say Joddojaaap & I9IDBSUOD }Sd UOTIES 39D 


S4INVUNOOD NOILVIWILSA.d SHGOHLYN “€ 


“"Y 9910U 
BIOS J[QLIILA B19) ‘UOT[HURYOS | ap saytUN say soino} Inod 
a9AIOSGO sIqeLIeA a[Nes ouN,p anb asodsip au uo nb Jssod 
-dns issne es UG *(Z'Z) 19 (['Z) suonenbs sap ‘mM JOUTUITS 
ynod uo ‘Isury ‘uonerndod ey ap saiiun soy soyno} Inod sjueys 
-U0d JUOS BseUUO][HULYD9,p sprod say anb ‘asatns eA tnb 
90 suep ‘1osoddns va uo ‘9y191[duIIs op suOsTeI sap INog 


“sojuepuodal 
-uou sgitun say Inod saginduit sinejea sap juos "7 Sa] NO 
1. S93! 
anes 
: ‘ I 
(77) = id 


ve RINK ze yim Pet * 
:soquenbueul SINB[VA SII queynduit uo nus}qo 
oli9 jnod suuskoul RP] Ip IMa}VUIT}sa sine Uf) “Ss9UITSS 
asuodal ep gufiqeqosd Rl Op VSIOAUI | kB ® Juepuodsel09 
spiod Sop JUOS : um yo UOTDEI9S Op aupiqeqord 1 (=) 9) SPU! al 
e juepuodsa1109 aseuUuo][MUeYoe <P sptod So] JUOS nr no 


@|qesoubl-uou esuodai-uou ep eouaseid Ua UONEWSE,P epoyJeW EU) :JuOWNeEg ort 


‘9.LO WIM epeuea ‘(ouejuQ) eMeNQ ‘epeurD onbnsneig ‘sodeugui sop sgidne sajgnbus.p sapoyisu sap UOIstAI “‘JUOWUNvE stoSueI{-uvaf | 


Uni n Mex 
(12) <r 
DEE Kan 
:sojuepuodal soyiun 
S$] JUvIgpuod us nua}go ang jnoed ‘uoneindod ev] op sI[1e1 
elisa no ‘Ay/‘'4 7" = Wt ‘ouuasoul e] op INayeUINSET 
"{ DOAB B9[91I09 39 
uOT]NULY | op SaytUN say $o}No} Inod daatasqo sfqeueA ouN 
sulow ne,p asodsip uo,nb asoddns uQ ‘C 9)0U 389 sjUep 
-uodgJ-uou sap uoT[HURYoE | 19 Y 9}0U 3sa syUepuoda sop 
uoTHULYE “J “JWOULI[Nes UOT[WUeYE,| op omed oun Inod 
IPAIOSQO 1S9_{ B[QULIPA PJ 19 F UOT[MULYSE UN sUUONDITIS UO 
‘Toate A Inog ‘gd uonetndod oureyiso oun nod J s[qvuea P| 
OP SUUDAOUL P| JOUINSS,p 9]U9} UO ‘aIAINS BA INb 39 sueq 


NOILVLON ‘7 


‘UOISSNOSIP 9AQI BUN JUITJUOS UOT}DIS 
SIQUWp ek] ‘UsWTeULJ ‘saJUspgogid suoNdes xnep xne 
SJLID9P SINIJVUIT|SS SJUdIIJJIP So] Joredwi0d op yueyouLIEd 
UONL[NUIS Op spNj9 ouN Pp syej[Nsal So] DILIDEp & 9s UOTIOES 
oulginbulo ey “aqeioust sed ysa,u osuodgi-uou ev] puenb 
sgsodoid uoneWNss.p spoyjeu ev] ajuasgid uO ‘UOT]O_S 
owennenb ve] Y ‘seguUOp Sap UOTNGLSIp ve] 39 BsuOdgI 
9p SUISTURIOU 9] JURUIZDUOD saseuodAY soyUaIaIJIP snos 
uonendod oun ,p suUdAOUI P| Op SINDJVWINSS SJUSIPJJIP 19) 
-Udseld & L19S UOTIDIS SUIAISION BT “UOTWeJOU op ned un zINp 
-ONUI UO 39 BWIQ{qGOId 9] JUIJap UO ‘UOTIES OUAIXNap P| V 
‘gqeioust sed 
3So,U osuodgJ-UOU ev] puenb sinaLa sap 9iI[eULIOU op esau} 
-OdAY | Op SUpIeA P| JOIJU9A op. juRyouLed o]duts spoyjout 
oun juowsles9 ssodoid uM ‘sINALIa sap UONNQMISIp P| 


Jatjiogds ap sed o31x9,U af[9 nb ay0s 3]]91 op ajINsUOD 3s9 
ayja ,nbsind 9ytyeu0u ap asaujodAy | & woddes sed a3snqo1 
UOTVUINSS Pp spoyyou oun ssodold Uo ‘aoe 399 sueq 
‘yuanonb 9] sed uonenduit | nod onb sayqru 
-OdSIp SIOJa}NO} JUOS BU UONIELIOD op sINajoRJ Sag “asuodgs 
op SWIsIURdSU 9] Inod a[Qpour suvs 39 OUTeWIOU ap ssey} 
-odAy J & ILINODaI sues s[qeioUsI sed jsa,u Inb asuodel-uou 
eR] Jed gipuasud sterq 9] o1Inpel Inod sarduits UodeLI09 ap 


-$INBJOVJ Sap JUdJUsaId (P66) [ePUIVS 19 OV] ‘UNoOoURY 


‘JeIQIUI.P 
aque ey yuenbrydut sjapouw 3] Inod sinowe sop syt[euLIOU 
OP IQRIJLIDA JUSWITIOIJJIP osaujodAy | Ins sgseq so douR]G 
-WDSIVIA OP WNWIXeU Np spoyjew ev] Jed uONeUINse | aonb 
SO BpoyJaW 99309 op syedioutid onbywo ey] ‘soyuenbuew 
SINICA Say Joynduut 39 (7G) SuLYOsarZ, 19 d0y ‘sooTUIeIH 
suep ‘sone aud ‘SUIWIOD soURTqUIOSIeIA Op WINUWIIxeUl 
Np spoyjeul vy] Jed UONeUINSA | JON}DOJJO ‘1QISIUT Pp O[QuLIeA 
RP] 39 Vsuods1 op sWIsIURDEUI 9] JUSWAJUTOfUOD JosI[gpoul 
nad uo ‘aulatqoid 99 JaUOD INO_g “[e}0} UN,p no suudAoUI 
oun.p uoNneWNss. suep jueOdUII sIeIq un JoIpuosua 
jned asuodg-uou ev] op a}dwi09 sua} sed ou ‘uoneNIS 9119} 
oun sueq ‘3[qQeioust-uou ssuodai-uou op yd30u09 np sou 
-10J UONIUTJap oun Inod (786) aN & PAOAUAI 4S9 INd}99] 
FY] ‘BSIDA-9OIA NO ZAII9 NUdADI UN yURAR SoUUOSIad so] 
anb oipuodal B souepUd} SUTOU JUO NUDAAI JIqQIeJ UN JURA 
souuosied say onb Jasoddns ap aystyeai ang nod [I ‘nusAdl J] 
dinseul uO ,nbsJo] ‘ajdwiaxea Jeg “a[qeiousI-uou ssuodgi-uoU 
ap souaseid ud yso uO Nb yp uO ‘4yUdWIATNES UOT[HUeYDE,| 
op omied oun inod anb agarasqo jso,u Inb ja aygnbua 
OUIQUI 3}]99 SUP SdINSOW 19J9IUI ,p sIQeUeA suN.p puadep 
ajygnbue oun suep ssuodgi op owistuesgu 3] onbsi0y] 


NOILONGOULNI ‘T 


‘uoleIgpuodal ‘uorssa1891 


ey ged uonejndut ‘uojewnyss.p suonenbg ‘soue[quiostesA op winwiIxeul ‘a[qesousI-uou asuodgJ-uON ‘Sq IO SLOW 


‘ajqeioust sed yso,u asuodgi-uou 


| puenb sina sap gyl[eulJou op ssayjodAy,| op duIpIeA e] JOYA op ueyoULIOd a[durts spoyyeul oun yUoWIaTeS9 asodoid 
UO ‘OLRD BUOJ] Op suUONR[NUIIS op USAOUI Ne D9NTRAP JS9 OPOYJOUI BT “SINILIO sop UONNQUIsIp e| JayIoeds op sed o8Ixa ,U 
9][9 Nb aps 9][9} op 931NNsUOd sa aT]o Nbsind 9}1TeuLIOU op ssaujodAy | e Woddei sed a\snqgo1 uONeUINsSS,.p spoyigu oUN 
asodoid uo ‘ajore Joo sueq J919]UT,p a[qeuRA P| JUeNbI|duL sfepour 9] Inod sinaLia sap P}I[eULIOU op s[QeIJUDA JUSUWATIOYJIp 
asguiodAy,| Ins agseq so sOURTQUIOSIRIA OP UINUITXPUL NP spoyjeul e] Jed uONeUITsa,| onb 3sa spoyeuI 9y}99 op ofedround 
enbynuo ey ‘souR[quIosIeJA op UINWITXeU NP spoyjou eT Jed UONeUITSI,| JoNjIaTJo Jo 1QI9IUI,P a[quURA eT 39 VsuOdaI 
OP SUISTULIOU 9] JUSUS}UTO[UOD JasI[gpou! ynod Uo ‘auIg[qoId 99 JoNUOD INOg ‘[e10} UN,p No suUdAOU! 9UN_,p UONLUINS | 
suep jueyodut steiq un JoIpuasua ynod asuodgs-uou ey] op a}duroo 11u9) sed ou ‘uoNenis o][9] oun sueq ‘s[quIOUsI 
-uou ssuodsi-uou ap dduasgid ud ys UO Nb jIp UO “JUOUIZTNEs UOT[HUeYDY,] op osed oun Inod anb agasasqo 3so,u Inb 
J9 a3gnbus sUIQUI 93}99 SUBP OgINSOUT 1919}UI,p 9[queA oUN,.p puedgp a}gnbus oun suep ssuodgi op auIsTueogUI 9] anbsIO] 


aWnsaa 


»LNOWNVdd SIOONVUA-NVAL 


d[qv10USI-uOoU dsu0d3Z.1-UOU op ddU9S91d Ud UOLVUIISI,p spoyjoW 9UL) 


Stl 


epeueg enbiysiels 
LSt-SpL “dd ‘Z .U ‘9 10 
000Z aiqwessap ‘ajanbua,p senbiuyoe | 


‘TOI-LSI ‘Op ‘Sotujamorg ‘one Xos UMOUY & JO 9dUasaId 
oy} Ul UONeUINsa oImdesol-simdeD (0661) ‘(WN ‘WALTON 


“OVE-BEE 
‘18 ‘uouvis0sspy jooysyvig uvoidamp ays fo joudnor ‘eyep 


SNSUID JOJ S[OPOU! JOLIA VBeIOAOD SWIOS (9861) ‘WM ‘MALTON 


“SLI-LOL ‘ph ‘uounI20SsP JoatsiIDIS uDdIuaup 
ay} fo jpunor “UOeNSIZ9I JO JUD}X9 OY} puUe so}eI YJeOp pue YUIG 
SuNeUINse Jo poyjoul & UO “(6h61) "AM ‘ONIN 9 “O'D ‘UVAAS 


‘O9E-ESE “Eh “OYldaworg 
(9861) HW “‘ATTIINONENO 


ISCIAUGD 
‘0% ‘algnbua,p sanbiuysay ‘ayajduioo a8epuos op sseq oun,p 
sed asodsip ou uo ,nbsioy uonetndod ap xnei0} ap 19 uoNe;ndod 
Pop a][!e) e[ op UONeUINSd | Inod sIstessaJ-aistes op sonbruyoaj, 
‘(p661) “VO ‘NMOU 1 “O'S “AANYAL “HA ‘MOOTIOd 


‘OPE-6ZE ‘Ob ‘S21ujaworg ‘syusutiedxe 
eaouiol pue sinjdeoos-sinjdeo ur saqeuea Arerixne jo 


asn OU (7861) ‘'f ‘STOHOIN 1° “d'f ‘SANIH “HM “MOOTIOd 


“8ET-S7Z “98 ‘MONIDID0SSP JDINSIIDIS UDI4aUy 
ay} fo jousnor ‘ainjny pue ‘juasaid ‘sed :suonendod ay1ptim 
pue ystj Joy syoyouresed o1ydesZouap Jo uoeWINse JO} soNstEIS 
[eaoulos pure ‘oinydesal ‘ainjdeo sulfapow (1661) ‘HM ‘NOOTIOd 

8t-1 
‘9 ‘DIS JO1g ysiung ‘day “eas UeULIOD 24) Wo prof] oy) OVI 
sored Bunok Jo uonessrumuyt Apreak oy, (9681) f'D'O ‘NASUALAd 


‘UONRUIT]SO UI SeIq UO Sa}0N 


“SEI-I ‘79 ‘sydousouopy aftjpjiy ‘suonetndod 

[eullue posojd uo ejep oinjdeo Joy aouUdIOJUL [PONSNIS (861) 
“Ud ‘NOSUFCNV 19 “DD “ALIHM “do SNVHN&N” “Td ‘SLLO 

"679-6£9 ‘ZS ‘Saldjamorg 

‘Ayloua80J2}0Y YIM sjapour simjdesai-sinjdeo pasofd om) Jopun 
AIW oneurereduoy (9661) ‘HM ‘MOOTIOd 3° ‘Tf “SIHYON 

2 ‘ql ‘uoISuryse A ‘2dIAIOS BuIpoday [eonsneIS 


‘ginqnousy jo juourredag ‘sq ‘0g Uodey jyeis ‘sJojeWNsg 
owesy eary pue afdniny oy) Jo MaIAoY “(p861) ‘d'f ‘NOTVAN 


uoeindod ap seiyiyo Sap ja aylj1e} B] BP UONEWIISA -eiNJUe 39 YOOI|Od ‘SeUule}H 


"887-787 ‘uouDIo0ssp 
JDIUSHDIS UDII4aU py ‘UO1JIAY SI1IS11D]G jo1I00G ay) fo ssuipaad0Lg 


‘sAOAINS owe s[dn[nw ul siojewNs| (8961) “aU ‘ANAT 
“SIT epNoD 
‘sinynousy Jo juowyedeg “s'Q ‘swIMey BuIpueg Jo siseg 
dy) UO souRpUNgY [MOJO AA SUNLTNITeD “(O€61) ‘O'd ‘NTOONIT 
"EO7-S8I “SUOS 2 ATLA UYOL 
YOK MON “(XOD “O'A PA) spoysayy Aaaung ssauisng ‘sKoains 
ssoulsng owey-afdninw (S661) ‘Wd “THOOA ¥ “S'd ‘LLOW 
‘OVI-EET “OL ‘Dyjaworg ‘sjusuIadxa 
ainjdes jo sishjeue Jeonsneis ay) UO (6861) (WY ‘SNIDDNH 
°C89-€99 ‘Lp ‘UO1IDIDOSSP JDI1ISIIDIS UDIIMaUP ay] 
fo JOUANOL ‘ISIZAIUN dU & WI JUWIZIeI[ dos ynOYyIM Sul[duies 
Jo uonezifesoues y (7$61) ‘f'd ‘NOSHWOHL }° “D'd ‘ZLIANOH 
‘811-66 ‘O ‘€ ‘9¢ ‘Bdyyung ‘suoneordde 
poiejes pue ASojopoyjour ourey opdnin “(pL61) O'H ‘AD LLUVH 
‘907- £07 
‘UOIDIIOSSP JOINSNDIS uUvdIIMaUp ‘UOljIasy soysyojg jo1209 
ays so ssuipaes01g ‘SKaains ouresy ajdniny (7961) ‘O'H ‘AD LLUVH 
“suOS 
2p ASTI UYOL :YIOX MAN ‘/ Croay], pun spoyjayy daaung ajdung 
(ES61) “O'M ‘MOGVW 29 “N’M “ZLIMYNH “H'W ‘NESNVH 
‘OST-SHC 
‘¢ ‘SONISIIDIG JDIIZO]OIT puv jojuawmuoauy “Ppoyjyoul surely [enp 
ay) Jo uoneordde ue :sjsou o[3ee peg [Nyssooons pue sanoe jo 
Joquinu ay) BuNeUNsy (48661) HM “MOOTIOd 19 “A'd “SANIVH 
"8816-18 ‘pz ‘atanbua, p sanbiuyra J 
‘uonendod ey ap saijjiyo so] 39 o][Fe} eB] JowNse inod sojdnjnu 
saseq ap UOsTeUIquIOD (88661) “HA “MOOTI1Od ?9 “dd ‘SANIVH 
“AVSIOAIUL] B}EIG BUL[OIVD YON ‘3eI0j}D0p 
ap asauL, ‘AZojopomapy aindesay-oinjded pue owes ofdnnyy 
Bulsp) sJojoweeg uonefndog sunewnsy (1661) “Ad ‘SANIVH 
“6VC-SVT 
‘UOIIDIIOSSP JDIYSIIDIS uDpIdaup UoOljIag soisIvIS, [v1I20 
ays fo sSupaaz04g ‘souresy Burddejioao om) WO poyoojas sotdures 
Joy sioyeunsg “(7L61) “AT “MALLSIAINANE 19 “WM YETINS 


vl 


"69I-LSI ‘81 ‘azanbua, p 
sanbluysa J. “USUIdIQUIOUYP-SNOs J] JOUTUITS Inod JUsWasUsde1 NP 
SOIJJIYO Sop JUSUTISsoIpoI ne UOTeoI[dde doAv aIstesal-sIstes Be] Op 


aple | 8 UONest[gpou ef Ins atydessorqrg (7661) ‘A'S ‘DYAANSIA 
‘C6T-6LT “7 “SOUSIIDIS 
jo1aif{[oE fo Jousnor *sayey§ peiwuy) 9U} Ul oIMy[NIUSe Joy sows 
surjdues (9861) ‘Vd “THDOA 1° “Ga ‘WYOLYOL “a ‘OSOdA 
‘6bb-Stb “98 ‘uMoNDIIOSSP JDILSIIDIS 
uvridaup ay} fo Jouanor ‘sue vale ue SUISN [e}0} UOTe[ndod 
e sunewns” (1661) ‘V INOUVD 1 “d’D ‘ANNA ENaAINVA 
“ESE-LVE “18 ‘UOLDIIOSSP JDIIISHDIG UDI14aUp 
ay] fo JDUANOL “SUOTBAIBSQO PdIa\sNj{O 9ABY SBdINOS YIOG Udy 
sjopour eimdesos-oimdeD (9861) ‘d ‘DA TVW 19 “d'D ‘NVMOO 


"suOS 2 AZTIM UYOL YOK MON 
‘UOnIps sUgi-¢ ‘sanbiuysay suyduvg (L161) ‘O'M ‘NVYHOOO 
“AYSIOAIUL) 3181S PMO] ‘1€10JDOP ap ase], “SADAING OUILIY 
aidniny jo suoneoyddy pue Arooyy, ($961) ‘SY ‘NVYHOOO 


“sonsneis 

UI UONBIITGN, BIUIOJITED Jo AjISIOATUL] ‘EIUIOJI[ED JO AjISIOAIU- 

‘sasnsusd = JeOIBOJOoZ +0} suoneoddy ym vuonnquisiq 
daWOadIadAH oy} Jo samadosg BWOg (S61) ‘O'd ‘NVNdVHO 

“917-107 

“Qp ‘soldjawmolg “jeullue [eNpIAIpU! pue ou11) Aq Area saniiqeqoid 

aimdeo uaym eyep oimdeooi-ainjdeo 10} ozis uone[ndod 
sunewnsg (Z661) “I'S ‘ONAL 1 “W'S “AaT “V ‘OVWHO 

“O0E-S6Z ‘TS ‘juawmasnunpy afypjiy fo jouanor “eyep Aouonbayy 
ainjdes ym souRpunqe [ewlue suneWwsg (8861) ‘V ‘OVHO 

"9€6-L26 ‘09 ‘ABoj00q 

‘sjeultue Suowe Area santtiqeqoid oinjdeo uaym ozis uonendod jo 
uoneunse Isnqoy (6461) ‘S°M ‘NOLYAAO 9 “do “WVHNUN 

"€£9-S79 ‘C9 ‘vylajaworg ‘speuttue 

suowre Area sontiiqeqoid ainjdeo uaym uone[ndod pasojo e Jo azis 
ay) Jo uoneUIns| (8/61) ‘S'M ‘NOLYAAO 39 “da “SWVHNUNG 

“AVSIOAIU() 9381G UOSIIO ‘Je1OJOOP op dsaU,L, ‘sjeWIUY 

suowy AeA soliyiqeqoig oinjdey usym soipmg oinjdea 
a[dninyy ut ezrg uonejndog Jo uonewmsy (761) ‘dM ‘NVHNUNd 

‘VCC-61T 

‘UOIIDIIOSSp JOIISIDIG uUDIIdaUp ‘UO1jIaS SoijsDoJg jo1905 

ay) fo s8uipaaz0ig “urewiop depJoao panes yA uoeUINso 
suey adninw (9161) “Td ‘GYOd “WU ‘WaAMOASOd 


“SE9-€79 “OH ‘SAldjamolg 
"s[opoul oinjdeseJ-o1mjded UI UOISSaIZ9I DNSIBOT (0661) W'S ‘OH TV 


aIHdVaOOI Tala 


“snsua_ oy} Jo neoing np 
SONA SI] JUSWAITeSSad9U Sed JUDIQIJoI DU Jd SINIINe Sap So]]9o 
JUOS SogUILIdX9 sana SY] “AJISIOAIUL) 9321S BUT[OIRD YON 
P| & SIWOI [IeAed] Op UO UN, Pp a1ped a] SUeP YDdIOYEI ]UNS 
-91d e] & gpsoooe Jorouvuly Indde | ap ‘AeaIng [eoIso;oeH 
'S'Q ‘UOISIAIG sooINOsoY [eoIso[oOlg ‘LSqgq owueis 
-O1d np alteuuonses ‘young ouNsiYD JUsWsTeS9 JUSIOIOUI 
-a1 sIneyne sa] ‘gsodxa | ap uoNeyUasgid e] 39 NUDUOD 9] 


vl 


Jaloljauie ,p stuuad yuo inb sanbsewisl ap jutolpe inayoepel 
UN jd INDOVpPal dB] JOIOIOWIII B JUOUUDT) sINayne say] 


SINGING NAINA 


"SoTquLIeAod 
Sop UONSUOJ Ud UOISNTOUI.p saiIqeqoid sa] JasI[apour 
yo yoTduroout a1g}9eIeD Np ajduI0d Jud) ossind Inb wore] 
-ndod ap saijjiyo sap uoleusa pp epoyyeur oun Jasodoid 
& Siatuaid say med Jua}dur09 syeI]Nse1 SON ‘sinof say sno} 
OP dI][eaI UN 38a sajsi] Sop yo[duOoUT a1Q}deIeD a] onb sored 
ajygnbua .p saysAyeue xne sopn saz ang juorelinod gsodxa 
juasgid 9] suep sagsi[n sapoyjsu say aonb suoKoI9 snoN 
“sinayeultysa sonbyanb 39 yusWeTNas sayapouwl 
xnap Inod uonrnulis ap apmd.p sjej[nse1 sap suojuss 
-91d snou ‘(/66[) Soule} SURP S9IOPISUOD 939 JUO SINdyeUT} 
-S9 SdI]Ne 39 SI[QPOU XNIIQUIOU ap IS SUIVJAT “UOISNIOUI .p 
saitiqeqoid say inod anbysiZo] uorsseiz9e1 op a[apour np 
sonquievied sap UONCUIT}SA Pp SOpoysUI Sop gjUasgid SUOAR 
SNON ‘SdllvjOsIe saseq ap 1d Sd}SI] Op soi suoT[MUeYDE,p 
uonouoj ua uonrindod ey op [e10} np 30 UoNe[ndod ex] 
OP oIIe) Ve] op sWINso UOSdWIOYL-ZNAIOH op Inayeunsd | op 
JUDUIOpUAI J] SIPNIS SUOAR snou ‘gsodxe JUasgid a] sue 


GYIVIWINOS ‘Fb 


‘said (1L6'0 ‘626'0) = (L000 1/S0‘0 x S60] € = $6'0) 
@ yoANnoN os sanbuidura ainyeanod ap seayiqeqoid 
sap uednyd ey ‘Wp yuowanNy “S6‘Q ep seid sadA}-sinala 
ston e@ yednyid ey nod yusanoy as sanburdura ainyaanod op 
saiiqeqoid say] onb yuawefes9 Ja}0u Y ‘odA}-IneLIa | op uoN 
-BUITSS DUO oUN ayUasaidal (7) Ud odAq-1na19 | op SATJeUI 
-1xoidde a[nuzi0j ev] anb anbiput e[a_ *[e10] np UOTDUO] US 
PSITPWLIOU INoyeuINs9 | op onburtdure adA}-y1e99 | ap syoo1d 
$91} So QUUSAOW SANLIOI V9UITSA OdAj-INaLIa | onb yuoW 
-g[e39 suonbrewial snon ‘a}UaWIsne UONeNdod ey] ap oIIIe) eT 
anb 3a ajuow3ne "d onb sinsouwl v yuonuTWIp odAj-InNeL | 
J9 JNLTOI SIvIq 9] ‘[elgues uq ‘uoNe[ndod ze] ap ay{Ie} e] INod 
QAIISGO SUOAR SnouU onb 20 ke Satqe{quas $y jUOS INb syeq[Ns 
-91 sop suoud}go snou ‘uoendod e] ap xnej0} soy Nog 


uonelndod ey ap [v}0} np uoVeUIS| TLE 


‘Opn oJ0U SUBP UOTeIEpIsuOd 
ua suid sed juos ou nb ‘arejoare aseq ev] op oseuUo[N 
-UBY99,P $INOD Soy Jed onbryead eT suep QUILLI}9D Iso Yd ap 
x1oyo a] ‘d op snajea opues3 oun dane “AV op UONeSTTNN | 
SUOPURUIWODEI SnoU ‘sUONR[NUIs sou sp nus} aIdUI0D 

‘quouraatoedsal ‘OOO I = N 3° OOE = N Anod 600‘0 39 ILZ‘0 
ap uos "a7 ins spuoj %C6 Op 9OULIJUOD ap o[[eAIOIUL,| Op 
SINLOANOD ap saiiqeqoad say ‘anuos Jeg *¢6‘0 op seydoid 
$a} JUOS “AJ INS 9PUOJ %C6 OP oURIJUOD ap S|[CAIOUT | 
9p sINOANOD ap saITIqeqoid sa ‘auIgUI aq] “aIqIeJ JUSWOAN 
-PJal jo oouargysIp ef ‘onbrtdure odAj-yWed9,| F SINLIgJUL 
1S9 “A Op deuINse odA}j-INII | ap BUUSAOUI ¥] IS OW 


000z a:quiacegp ‘a}enbua,p senbiuyoe | 


£760 Ly¥6‘0 Lv6‘0 vs60 6+6'0 7560 INYOANOD 
6000 110°0 Z10°0 L100 0200 1Z0°0 eguinsa adA}-Inaua,| ap suUaKo 
0100 110°0 Z10°0 L100 020°0 1Z0°0 suoneunsa sap odA}-yreaq 
000°T 000°T 000°T 100°1 100°T 100°T suoneunsa sap auuakoyy OL 
$960 L960 $960 1S6‘0 0560 7560 aInaANoD 
910°0 8100 610°0 620'0 ve0'0 S£0'0 guise adA}-snaUa,| ap UUSKO] 
S100 810°0 610°0 8200 €£0°0 S€0°0 suoneumnss sap adAq-yreoq 
100°T 100°T 100°I 100°1 Z00'T 700'T suoneulnse sap auuakoyy 60 
$560 1S6‘0 Lv6‘0 7560 0S6‘0 €56°0 SINYIANOD 
8Z0°0 ZE0‘0 v£0'0 1500 1900 790'0 aguimse adA}-snaL9,| ap suUsKo 
LZ0'0 7£0°0 v£0'0 0S0°0 790°0 ¢90°0 suoneultisa sap odAj-uesq 
100°T Z00°T Z00°T c00'T L00‘I 800°T suoneUlyse sap auuaroy 0 
$560 6S6°0 7S6'0 1€6°0 9760 S£6°0 SINLOANOD 
0r0'0 840'0 1S0°0 >L0‘0 6800 +60°0 aguutyse adA}-InaLIa | ap uUaKo|, 
1700 6+0'0 7S0°0 8L0°0 760'0 860'0 suoneunsa sap od(}-yeog 
Z00'I €00'I €00'T 900'T 600'T O10'I suoneUmnsa sap suUucko] €‘0- 
Aned Ae Hed yaa Doon yan ey 
000 1=N OO€ =N 
Z a[Qpour 9] snod { uojas a[[sY99,| B sas oseuUOT[HULYD9-snos Jed uoNL[Ndod ve] ap [e101 np suoneMNsy 
g neayqe 
1S6‘0 6>6'0 9660 0£6'0 6760 0£6'0 QINLOANOD 
960'0 650°0 0L0‘0 980'0 8710 LLZ‘0 aguinse adA}-Inaila | ap sUUsKO|] 
Lv0'0 990°0 760'0 060°0 42) 0) SIs‘0 suoneumse sap adAj-yreog 
900°I 600'T £101 EL0'T O£0'T 790'I suoneuinsa sap cuuaXoW] OT 
SS6°0 $560 0s6‘0 Sv6°0 1S6‘0 0s6‘0 aINYOANOD 
9€0°0 Sr0'0 1S0'0 0L0°0 60°0 €€7'°0 aguinsa adAj-snaLia,| ap auUsKoW, 
LE0°0 S+0'0 LS0°0 ZL0°0 vOr‘o 9850 suoneuinse sap adfj-presgq 
Z00'T €00°I 00'I 600'I 810'T €SO'l suoneunsa sap suuako C0 
9r6'0 760 >r6‘0 1S6‘0 S60 1S6‘0 AINY2ANOD 
€£0'0 8E0°0 0r0'0 090'0 ZL0°0 1800 a9uinse adA}-Inaiis | ap suUaAO| 
€£0°0 6£0'0 1700 6S0°0 0L0‘0 080‘0 suoneumnse sap adiq-yeog 
100‘I 100‘T 100‘T 800'I ZIO'T E10! suOTJeUINsa sap auuakop] =) 
760 7r6'0 9460 6+6°0 1S6‘0 €56°0 INL2ANOD 
€£0'0 6£0°0 170'0 190°0 ZL0‘0 9L0°0 apuinse adA}-InaLa,| ap euUaho] 
S€0'0 0v0‘0 Zr0'0 790'0 €L0°0 LLO‘O suoneumnsa sap adAj-yesg 
100°T Z00'T Z00'T 100'I €00°T 700'I SUOHRUISS sap suUakOWY ¢‘0- 
Peron Bia ih FOE & Oia Aik ay 
000 1=N 00€ =N 


[ 2[gpouwr af inod J uoyjas aT[ayo9,| B sastul a8euUOT[NUeYDI-snos Jed UoNe[Ndod e] ap [e103 Np suoneWINS| 
p neaquy 


“oUSUIS Ne 
uone[ndod ey] ap ayjte} ey onb ainsoul & JUONUTWIp SOANe[OI 
sodA}-sInaLa Sal ‘ajeiguas UoddSe] aq °%6Z7 ep UONNUTIWIpP 
aun ‘Sz7O‘0 & SE0‘O OP Juelor odAj-yWe99,] JossIeq je] 
WOT B aurejoale aseq op UOT{NULYDE | JoJUDWISNe Pp We] YT 
‘%pyl Op uononpas oun ‘O¢'O & SE0'0 Op Jue[ar adAI-11e99 | 
Jossteq ej %¢ ap allejogre aseq op UO][WUeYOS UN ,p UOIS 
“NYOUL,| ‘O00 I = N enbsioy -%g¢ ep UONONpgl oun “HOO 
® LL0‘O ep enurunp jneper adAj-11e99 | “%OZ OP sle[O9Ie 
gseq op UOT[MUReYDE UN ast[NN UO .NbsIOT °%EZ op UONONpel 
aun “6C0'0 & LLO‘O ep Juepes odAj-yedg | Jassieq We} %C 


uolyeindod ap saijiyo sep je ayjie} e] ap UONeWUSA :ejNJUe }8 YIO||Od ‘SeuleH 


Op aslejosse aseq op UOT[HUeYD UN,p UOISN[OUI.| ‘OOE=N 
NO | 2[Qpour 9] suep ‘o[dutoxa seg ‘ajuoursne “d ameogue 
a Rl Op UOISNIOUI .p ayt[Iqeqod ve] onb oinsou! eB oNUTUIp 
“aN ap adAj-11e99,] ‘o1puaye A,s wWeanod uo suIUIOD 
-gquowdne uoyendod ey] op Tre) 
eR] aonb onsoul B ONUIWIP Jue[oI sterq oj ‘sntd aq “7 9]Qpoul 
a] suep ajuow3ne “d onb ainsow ev onurUtp a suep 
sIviq I] ‘N op UONeWINss oUUOG 9UN WUINOJ an ‘a1uod 
seg ‘uone[ndod zy op ayfrei aresa ey] JUoWe[qetoidde ous 
-snos a7 anb suoasasqgo snon ‘A Inod gudoidde inayeur 
-yso un sed qsa,u a7 Guanbgsuoo seg “a[qeueaod eB] ap 
UOTIOUOJ 1S9 UOISNTOUL,p sUTIqeqosd eI ‘z afepour aj nog 


cvl 


‘said (96'0 ‘p6'0) = (1000 ¥/S0'0 x $60] € = $60) 
B® yUoANON as sanbiuidura simseanod ap sgi[iqeqoid 
$2] S9INO} YIP JUOWIATINY °$6‘0 ap seid sadAj-sinoL SION B 
$9}No} JUBANON as sanbutdurs sinyaANod ap saiyiqeqold so] 
onb suonbseulal snou ‘aulgu oq *S}e[NSaI suOg op sUUOp 
suost{n snou aonb adAj-inaua,| ap uoNeUINsd | onb onbiput 
RPO ‘suoteUIsea sop odAj-yWe99,[ op oydoid yso gUINSE 
od4}-1199 | op ouUcAOU eT aonb suonbseursl snou ‘] s[gpoul 
3] INOg ‘OUIIUT JSso SUOTIBUITSA Sa] _$9jNo} Inod sIvIq J] 
“HOT anb soevorya issne sgid nod ke 1s9 aN INBeUINSE T 
“Q[QPOUL 99 SUep aT[NU Iso INgTeA eT JOP ‘ 23a ‘ ‘2g «0 
sonouresed say Joutse eB yas [I nbstnd gy1oevotJo os op prod 
ny anayeunsa TJ “Pay anb syey[nsau sIna[[Iow! op suuop [I,nb 30 
B OUOP pUudNe,s UO Ja (UISID}Og-U[OOUT]) VOURTQUIASILIA Op 
WINUWITXeU Np INa}eUINss,| ap syooid san isa Ay uewdeyD 
ap INaeUINS,| ‘sed 99 SUR “aTquLIeAOD BI op Sed yuopuod 
-9P dU UOISNIOUI.p saypiqeqoid sot ‘[ e[epoul of sueq 


uonejndod ey ap ayfie} e] ap UOHVUIS| T[°L"¢ 


suOISn]IUOD *L'¢ 


6760 6760 6460 600°0 amnywaanod 


5000'0 £000‘0 8000‘0 L00'0 o8equsomnod us YW 


N ded agstalp Inayewyse,| 
ap guINsa odA}-y1e99,| op UUDdAOJI 


ZZ0'0 L700 8Z0°0 8100 N Jed gstalp suonewnsa sap odAj-yre9g 
100'l [00'T I00‘T 1760 AN Jed a9siAip suoneurnss sap suuahoy] 


1Z0‘0 920°0 Lz0°0 S100 


000 T=N 
1S60 ¥S60 €S6'°0 ILZ0 amnyaanod 
z00°0 2000 £€00°0 L00‘0 asejusoinod ua YWOdY 


A Jed o9statp Inajeurnsa | 


0r0'0 800 ZS0'0 820'0 i aaa caieiso 1 REGEN ON 


Or0'0 6F0'0 7ZS0'O ZE0‘0 AN sed gstaIp suoneunse sop adAj-yre9g 
€00'I SOOT 900° 7760 A Jed s9starp suoneunsa sop ouuako 


Ca WOK ov HOW 00€ = N 
Z 3[Qpoul 2g] nod uoynejndod ez] ap aI[1e) e] ap suonewNs| 
€ nesqey 


9560 8660 6S60 S60 
1000 1000 100.0 100‘0 


dINYeANoD 
adejusoinod us YWOAY 


N Jed 29stalp InayeUurNsa | 
op guns 3dA}-ye99,| op ouUUdAO|] 


S00 O€00 SE0'O I€0‘O N Jed gstaIp suoneumNse sap odfy-yeog 
ZOO‘ ZOOL €00'I 000‘! AN Jed a9stAIp suoneumnse sop suuako] 


Sz0'0 O£0°0 ¥E0'O E00 


000 1=N 
0S6'0 LS60 SS60 Lr6‘0 amnyaanoD 
z00°0 ¥00'0 900°0 £000 asejusoinod us YWOAY 


N Jed a9stalp Inayeunsa | 
ap suns odA}-ye99,| op uusKhoJY 


8r0°0 6500 LLO‘O 6500 N Jed gstaip suoneumnse sap adfq-ye9g 
P00'T LOOT II0‘I 666°0 N Jed agsiAIp suoneunse sop ouusko 
Oy ON oN 2S) O0f=N 
[ 9[gpowi 9] Inod uoyjejndod ve] op ayre3 e| ap suoneunsq 
¢ nesqeL 


Lv0'0 6S0°0 7400 6S0°0 


LVL 


‘WUdUIDATIDNdsal ¢ 19 p XNITqQe} So] SUP Sg}UaSaId JUOS 
Z 12 | SaTQpour sa] Inod syey[nsgi saT “[e301 9] Inod %¢6 ap 
DOULIJUOD OP SI|[PAIOIUL SOP SINIOANOD ap soqyiqeqoid so] 
SUO|eUSIS Snou ‘UIjUq “oNNe,] e UONNEdel ouN.p JUasUeYO 
XNv}0} Saq onb sored sageusIs JUOS 19-Sa][9D “sodgstTeWLIOU 
SINIVA S9d ap DUUdAOUI I DJINsSUd SUO[NO[Vd snou 39 
‘uonnedas ey] Inod [e}0} 9] Jed S9gSTAIpP [%I0} NP INdJeUIT}SO | 
ap soullysa odAj-INeLIa | SUOTNITed snou ‘uoNHeder onbeys 
Inod ‘auIgUI aq] “Sa9STTEULIOU SUOTIBUIT}SA S20 ap sadAj-sye99 
So] 19 OUUdAOU RT SIOTe suOC[NI[eo snoN “YZ ‘uoKNedeI 
op [e10} uos Jed uONeUWNsd,| SUOSIAIpP snou ‘uonNedsl 
onbeyp Inod jp yuawanny ‘uonnedal ay99 Inod ({) [e101 
IVIA NP UOTOUOJ UD JI[OYDI,] B SOSTU JULI9 SUOTILUITISS SOT 
‘SINDJCUIT]SS SAP SaguIT}sa sadA}-sINoLI9 sap suUdAoW PT] onb 
OUIQUI Op SUONeUINSS sop odA}-11e99 | 19 OUUDAOU BT SUOU 
-NsSgi snON ‘onbiygwiesed uosreurquios anbeyd e yuepuod 
-Sa109 suonnedsl QOO [ Sey Inod souusXou ap SULIOJ snos 
sal]qeje soureumuos sanbysiyejs soguUOp sep suoUasgid snou 
‘uonendod ve] ap [e}0) np suoneUNsea say Inod ‘augUI oq 
‘UdWIaATIDOdsal ‘¢ 19 Z XNITqQe} So] SURP Z 19 | SATQPou! 
Sa] Inod sjey[nsgi sap suojuasgid snoN’ ‘A uone[ndod xy] 
OP Ile} VP] UOTAS SdgSTTPULIOU $9}NO} JUOS SAINSOUI $9D °%66 
Op sdULIJUOD ap o][eAJojUI UN Inod sanburdwia sinyeAnod 
ap saiiqeqolid sa] 3a asejUsINod ud (YOY) Saueyel 
suusfowl onbyeipenb inoue, op salvo ouToRI eT ‘sInd} 
-PUII]SS Sap S99UIT]SS sadA}-sINdLIa Sop SUUDAOUL PT ‘SUOTIEUI 
-11s9 sop odA}-171899 | 39 QUUDAOUI BT SUOUNSEI SNoU ‘aTQpour 
onbeyo nog ~ d onbeyo nod oped auoy op suonnedel 
000 I Sol 30° d op sineyea anenb xne juepuodsa09 
suonnedsl OOO - Se] Inod sarjqey9 souudAoW sap suojuaseid 
snou ‘uonendod vy] ap aye} ve] ap SUOTeWINSE Say INnog 


SaIIVUIWIOS SaNnbYSHeIG 9°¢ 


“SUOIOPISUOD snou anb sa[epoul sey Inod InayeUIT}se | 
op onbuiduss soueLea ev] op ayoold sau} Jsd SoUITISO DOURLIVA 
eR] onb suonbseulal snou ‘UOISSIWIO 31199 DISTRI, “QOURLIVA 
R] OP UONLUINSS aIJOU SULP SIQUIOU 99 snfoUT sed suOAR .U 
SnON ‘wed ong wesaop ‘ar ‘4 - “x yueyodutos “(7 1) 19 (QI) 
suonenbs sap aiquiswl puoses 9] ‘Isury */4 ‘4 ap ayooid yso 
(11) ua "x 39 ‘2 ap ayooid isa ‘x ‘uanbgsuos seg “NprlAIpuI 
onbeyp inod opuris jso sajst] Sop 9uUN SUIOW Ne suep UOISN{D 
-UI Pp ay[iqeqold vy ‘aud Jeg ‘sayed seq JUOS SagIEpIsUOD 
Yd ap sinajea so] ‘uOne[NUIs op spnig onou sueq 


ANIJVUIYSI | IP IQUIIYSS BIULLIVA S°E 


“snqout sed ouop quos ou "x Inod syey[nsgu so] ‘outepogie 
aseq P| Op SUOT[HURYD sop sjusuloustosual op snjd ua Sensi] 
SOP SJUSUIOUSIOSUdI Sop Injou “YZ Inayewinsa,j onbsind “7 
aonb spueis snd dnoonveg souruea oun J0Ae yeraap ” x “’d 
ained oun inog ‘anbyeid ey suep ained juowrenbidA} 1s9 
Vd ‘JgYO dINOO SallejOgTe SJUDUISas sap Jo[du09 JUSUaIQUIOUSp 
a] onbsing ‘WWOWs[Nes sIIe[OgIe aseq LI AP UOT[NUeYDS,| 
Ins gpuoj udsdWOY]-ZUAIOH ap AN}euIN}sd,| Op 113e,S [J 


Vq__,,arejoage aseq e| ap uoynueyog,, 3 / 


ie A 


000zZ e:qwedep ‘e}enbusa,p senbiuyos | 


Jed guuop 1so_{ op ueyd oy 

ee eee UOU INSJeWINSS oI]Ne UL) “IDI sNToUT sed yUOs ou 
a Inod sjey[nsai sx] *Z a[Qpour sj snod gore 

-de ay guinsa-snos e nb aN op pusedap (ee TA op JUSWIOp 
“USI FT] “UOT[HURYSE»,| SuEp sasnour ‘4 sap UoTTHUeYD9 ,| 


ap suuokour ef yso “€ no ‘gugpisuoo quaurateZe 3s0 


Vy« 
£ d‘sHD A 
INgyewWysa "J 
ws ©) 
onb suLIO} dUIQUI BI SNOs aJUaSaId 9s ‘ N ‘uoneindod PL op 
][!@1 B] op UOTeUIT)sa | ‘oUIQUI 9q *(G[) Ua a1uljep ‘yy O9AR 
(QT) onb su0j suIQUI eB] SnOs ajUasaId as . A ‘uonernd 
-od eI] op [e}0} np UONeUINsa J “aIST] onbeyo ap suoT[NUeYDE 
-SNOs Sap Ins segpuoj uoTe[Ndod ve] ap [e}10} np suoneul 
-11S9,.p sed suoJ9pIsuod ou SNON ‘0Z‘0 19 S0‘0 ‘0 = "d sally 
-Iqeqold sop 90Ae SQUUOT[NULYDE JUOS dITeTOSIe aseq oUN.p 
SJUDUIZ[9 SY] “OISI] RP] OP SJUDUWIITE Sa] Sno} Inod agasasqo Isa 
asuodal ve] jonba] suep sed 9] suoJgpisuod snou ‘asuodai ap 
g[qeuea oun,p uone[ndod ey ap [e10) np uoNeUINsa | INog 
‘Z UOTIOVS PT B UOTISaNb js< [I 
JUOP SQUIso UOsdWOY[-ZNAIOH op SsindjewiNse so] JUSUIZ] 
-839 SUOISPISUOD SNON “aite;ogIe eseq B] Op uoTTWURYD9 | 
ap uoneuojur,y sed astpun.uya Q= “d= %J aonb osoddns 
InayeUINsa 39D ‘(g) ue guuop “Yay ‘ueudeyD op inoieu 
-11$9,] SUOI9pIsUOd snou ‘uoNeNdod x] ap sTIIe} eB] INog 


020 ‘so‘o 0 = Vd Oye HO 


SANICWHSY pe 


‘O[IeD dU op suoNnnadal YOO | SUOI9UaS 
snou ‘enbiyguresed uosteurquios anbeyo inog ‘9z‘g='d 
sed of nod snssad0id a] suojadel snoN “Pp ollepogie 
aseq eb] & jUoUUSTIedde Inb sjudWg[9 so] SUOTJNUSPI SsNnoU 
‘C00 = °d op opie. e ‘uug ‘/arst] e[ suep jUsUIgI9 oUaI-7 
QJ ainjout inod “d giyiqeqoid ey suosyun snon “g 19 
'g saist] xne juouuaredde nb sguun say aynsus (suornuspr) 
«SUOI9UIS» SNON ‘Ay “ST = 2 Inod suoisuawtp xnap e onbruw 
“YLeSO] e[euLiou uoNNgINsIp e] op ople,| B (A “x) ploge.P 
SUOIQUIS SNOU ‘sNSsap-I9d Sa[QPOU! 9ZIas Sap UNdeY INOg 


saguUOp Sap UOKeIIUIH «¢°E 

‘a ya 'g saistl sap 

sJUdWIOUSIasUaI S[nas sap uONesI[NN,| e puodsaos Q = "d 

‘IT ‘OZ‘0 32 SO‘0 ‘0 sed saguuop "d sinajea sion suosap 
-ISUOD SNOU ‘sa[apoUl QI = ZX PX Z Sap UNoVYyo nog 


060 86890 8£89'0 80 8ZrS‘0- 80 8ZLPrS‘0- C 


SLO ‘0 ¢0 0 0 0 0 l 
tg at 
Os 1) eyed layed ‘tg “My 8g fy apo 
‘“(m D-I 


2[Qpoul ap sasguresed sap aIreWUOS 
[ neaiqey, 


"| nealqe) a] suep soguinsel 
yUos sajgpoul xnap set Inod ‘gd 30 ‘9 op solerpnonsed 
SINICA SIT “°Z J[QpOur a] suUeP VESI[NN ISO UONLISI 99D 
4d, — [) - [ Jed aguuop js9 says] xnop sap oun suTow ne 


SUP UOISNISUL,p ssouRYo soz sioye “g 19 'g sayst] say anod 
SOUIQUI S93] JUOS UOISNIOUI.p soit[Iqeqoid saj anb suosoddns 
snou Ig ‘f 9SI[ ek] suep osnfour ong,p d sytIqeqoid 
aun ayoduios suuakoul a]qeLeAod ap INg[eA ¥B 9IUN,| sIOTe 


eg NES 


suosod snou Is ‘snssap-19 sapt,| SUOpUs}a SNON 
A 
Q]QPOW 9] SUP UONRIAI 91399 suOsT[N snON ‘falsity eT suLp 
SNIOUT 3119,P SOOULYD AP BOE ¥ JUSUI]D 199 ‘...a'J- =D IS 


+) 20 - 


(92d + 'o) dxo+ | 


q=f = 
(ae +‘v)dxo  @ 


pees 
1S9 9ISI] SWIZI-/e] suep 
SNJOU! WOs USUISTP 199 onb gyTIqeqoid ey] *..,a suuskow 
Ingjea ek] Jed sguUOp s[GeIIRAOD Op INe[eA eB WOUIZTS 
un SSS, ‘oo? = L, 217 = (‘x)q onb suojaddey 
“ag Jo “My aq « “in ep xXIoYyd syusIgJIp e Uepuod 
-S91109 S3[QPOU XNop sUOIEpIsUOD snou ““d 19 AV ap snssap 
-19 sonbijouieied sase[se1 g = pxZ Sep uNoeYO nog 
‘asuodal op d[qeLIeA FP] & JUDWID}OAIIP Val] 
4S9 UOISN[OUI .p ay[Iqeqoid ey onb yuenbrpur ‘'4 = ‘x ve puod 
-S09 | = “'d ‘oy ‘a[qeleaod eI 39 asuodgl ap a[qeueA P| 
aus oyejsed yo sAntsod ‘ay[nu ‘aaAnesou UOTIe[aLIOD oun e 
yuepuodsa1ios | 19 ¢‘Q ‘0'0 ‘¢‘0- = d suost[Nn snoN ‘900 I 
no QO€=N Ssuosod snou ‘ay ‘uoyefndod ev] ap ayyrei ey] 
Inog ‘sanbuygwiesed sinafea SasIIAIp SUOISPISUOD SNON 


sarjguVied sap aselsay 7° 


“SIIe[OSIe aSeq ¥] op UOT[NURYDE,| sUBp 
19 “g 91ST] Be] SUBP UOISNIOUT ,p 1819 UOS ap o]URpusdgpuT 1s9 
'¢ alist] B[ SuUep SNIOUT IOS JUOWUE[9 OUIaI-7 9] anb ayyIqeqoud e] 
anb suosoddns snoy “g ‘'g = f19 NV“ ‘[ = 2 anod({) enbn 
-SIZO] UOISSAIZ9I Op B[apoul 9] Jed asguUOp js9 o)SI] GUII-L 
B] SUP SNOUT IOS JUOWIET9 SWIAI-7 9] aonb ayjiqeqoid eT] 
“SNPIAIPUI saz Sno} Inod anbyuapt 
aesoddns yso "d no ‘oirejogse aseq ev] ap uoT[nueYse,] 
suep onbuosjenb yuauigja uN.p UOrsNoUl.p sit[Iqeqoid 
ey ‘"d wos ‘altejogse juawi3es onbeyo op ules ne soqtun 
S9] SAIOSGO UO Jd oITe[OVIe aseq LI OP sJUOUWSES op UOTIN 
-ueyd9 UN aI UG “aenUS UONR[Ndod v| s1ANOd aa nb suas 
99 ua aja[dui0o sasoddns jsa airejogie aseq BT “PY ‘aJle[oose 
aseq oun yo “g 30 'g ‘soyuepuedgpur sayst] xnop ajstxo 
I,nb suosoddng ‘jy a][re1 op uoneindod sun suolgpisuo_D 
‘(1+(1-2)"d]3oy=“"d onb szanuow ap afqissod jso |] 
““d UOTe[ZLOS oun Jo SarTeWUN ssoURLRA Sap ‘saT[NU SsuUUsAOUL 
DOAR SUOISUSUIP XNop & Sa[eWLIOU SolTOJeg[e Sa[qeLieA sap 
quos ‘a yo ‘n no ‘, a aumut0d ‘A ya, 2 SUILUOD ‘x suoigues 
SNON ‘So[qeleAos sop onbujewiAse uoNNqUysIp sun aout 
-pe & uodSe} ap agst[NN Jso onbrwumyLeso] ayeuULOU VOTING 
-sip ey “ d wuonR[alio0s dene onbrmyueso] speutiou uoNNg 
-lsip oun quaains ‘4 ya ‘x onb suosoddns snou ‘snid oq 
“4 gsuodal ap d[QeLRA B] DOAR 99[91109 og nad s[qeLIeAOD 
vj anb suosoddns snou ‘juowiouarxneg “x a]qeLeAos oun .p 


uonejndod ap sasyjiyo sep ja alle} B| BaP UONEWIISY :ejNjUed }@ YOO||Od ‘SeUIeH Ort 


juapusdgp sais] xnop inod uotsny[out.p sayyiqeqoid say aonb 
gsoddns js<a |] ‘UaWIa[Nas Ssa[Qpoul $99 ap xnep nod s[Iejgp 
Sap lol suojuasaid SNON “sUdIOJJIP SoTQ@pow ySurA-onenb 
QIapIsuod & (£66]) SouleH ‘[e10)] NP sinajyeWIsa sap 39 
uonejndod ej ap ajjrei ey op sayaudoad sgj JoIpnie .p UY 


apnz9,, ap sasagyjodAyy ["¢ 


NOILVTOWIS AG aadNLaA *€ 


“(€1) ua stugp jos { 39 g 19 (1) ua atuap isa "uw NO 


lag ag ay PSS Y) 


Gad 7 coast ) 
1 1 = AIA 
(u- 1),4 a 


red sguuop so 4" 47 ap sour ev] op UONeUITSo ou 
woyTHUeYo? 3 1 


x j 
(91) Sf at pee 


yso uoneindod vy] 
ap [10} J] Inod guise uosdwoY]-ZWAIOH op Inoyeuyse un 
‘seo 90 SUR “JUDWISATDEdSaI ‘so}IUN ,p UOT[MURYDE-snos UN 
Inod jUdWIZ[Nas NO sajsI] Sap SjUN sayz $9]N0} Inod aga.tasqo 
ysa ‘€ aonb uojas (pI) ua no (p) ua atusp iso ‘x no 


(SI) ‘Cua de Bey 


91IP-R-1S9 9 “NPIAIPU! UIQI-7 
3] Inod sawiNse uOIsN[oUI.p sy[Iqeqoid ev] e Nal] ouUOp 
DIIV[OIIL ISeq LI OP UONPULIOJUI | Op UOISNIOUT J “allejogue 
aseq SUN.p SJUDWIZES Op olOJegTe UOT[NULYDS UN,p syUdUT 
' -g[9 saq sno} Inod safqruodsip juos ‘4 sinayea sap ‘(saisty 
Sop SUO]T[HURYDE-sNOs Sa] NO) saysI] sap seyuN so] inod 
'€ singjea op snid ua ‘fanbay suep seo 9] suoJepIsuoD 


$9}SI] IP }9 SIITBOIIE SaseEq 
ap apie | & uoKejndod ey ap [e}0} Np UOHeUINS| p°7Z 


"Sd]SI| SOp SUOT[URYDE-snos sap Inod jusuIENb 
-Iun Sa[qiuodsip jUOSs syUdUIBUBIOSUaI sop onbsiIoOy] IUIJap 
ASSIL] 98 a[qe[quias InsjeuNsa uQ “¢g 10 'g saseq so] suep 
sanbrun syuowgya “Way + “War + “Ay aISIXO [TT ‘soueSowoy 
JUOS UOISNOU! .p sayt{Iqeqoid say] anbsso] yuowonbrun apryea 
1S9 INDJLUITSI 399 ‘SIOJ BUN BIODU *d}SI] ap UOTTHURYDE-snos 
UN SUIOU Ne SULP SNIOUI s}USWgI9 sa] Inod sasuodai sap 
suusXow P| Jed gijdnyjnu ueurdeys op Inoyeunsa | 1s9 nb 

ines Iar + Ne * 


“ is viene, 
yso ‘Xx “UOTIL] 


-ndod RT 9p [210] Np InoyewINss one UN ‘sed 9d SUG 


6El 


‘(Z[) ua (pf) JUeMNsqns us { Inod 
deUINso UOSdWIOY]-ZWAIOH op UONeUINsS oUN JUIT}GO UG 


(1) 


}S9 Sd]SI] XNOp Sap oUN SUIOUW Ne sup SNOUT 
LOS NPIAIPUI SWII-7 J] anb aguutysa aytfiqeqoid vy ‘soyuepuod 
-9pul sagsoddns juos sajst] say anbsing “Sagurquuod saysl] op 
aseq OUN ,p UOT[HURYSE [Ns UN JOIN op nal] ne ast] onbeyo 
9p sUOT[HURYSE sep suoIN snou anb Ja}0U Y “UOT[HURYDE 
-Snos 9] SURP SNOUT og,p (# N/ {u) sayes9 saoueys sop 
UO ‘g O}SIT BP] Op SJUDWZTE Sat sno} “uanbgsuOd Jeg ‘afdurts 
aojyege oseuuol[yuUeyos Jed uo[[WueYyoe-snos un oI} 
Ud UO ‘d}SI[ DUN SUBP SNOUT SNPIAIpUI Sap STO o oun ‘stojayno], 
‘( aseq Be] 19 2 NPIAIpUr.] UOTes quaLIeA “"d ‘uOIsNTOUT,p 
Ssuutaeqord Sg] ‘UOHONNsUOd Ine Jed oq “Juoweatoedsar 
‘“o 49 'g saisi] sap soyrun fy yo fu ap Juaula[nas anoreale 
uoT[MURYDE UN Inod saqruodsip quos ‘A say yenbsy suep seo 
3] SUOINPISUOD ‘sajSI] Sop SazTUN Se] $a}no} Inod sagatasqo 
juouloiessaogu sed yuos ou ‘4 say ‘anbreid ey sueq 
“sajuepuddgpul sojst] 
¥ JoIOdIOOUI B UOSeJ ap JUSUTIOvJ JUSPUIII,S SdapI sad 
"9 INs soNTeAg OUUDISSaY SOLU L[ OP SSIOAUI,| ISO (Q){ 19 


ge Oe |r. 
(€1) 7 ; = 
(Que 4 : 
no 
(Q) I='W pis 
AO). CE tieenea a =( "DA 
ae Se (9) u-1),4 < 40M 


SULIOJ BP] SNOS ajUasgid as 39 JO[ART, op poyyou! eT Op 
apie | e saAuap sa 1 47 ap soueuea P| op uoNeUINSS ou: 


aH. 


(9) % tn 
“ = tH 


iso uoNendod zy] ap [e101 
Np uoneuiyss oun ‘g Jed suse Ja NuUODUI Jso Q onbs1O'T 


(@) t= 'W ety 
P7eN ET) a ead de, 
((@)u- 1),4 4 rine 


Acoree ah OP SOUINSS sOURLIA PI ‘(/L6[) URIYDOD soude .q 


(9) '% I5N 
; L-H 
(Z1) Ve Cee ari 
uosdwioy.[-ZiAIOY op Inoyewiy 
-S$9,[ 389 ‘4 ‘uone[ndod ve] ap [e}0} np UONeUINSs ouN ‘nuUOS 
9 1nog “g 10 'g sowuepuadgput says] xnep ap s}UUIZTE Soy 
sno} Inod sajqruodstp quos ‘4 sinapea sap onb suosoddng 


$9}SI| 
op opie] & uoKelndod ef op [e}0} np UoHeWSy ¢€°7 


000z aiqwesep ‘ajenbue,p senbiuyoe 


‘olgz op aysoiddes as (/u' x - “a) ssoje 
‘sopueiz quos ‘x 39 ‘x anb ja anad ysa “d ts ‘stozaynoy, 
-/'y #"¥ Quow8es oulgui ne juouUonedde 7 19 1 onbsi0] 


‘sury “('u- 1)'d+'w='u 19 (p) ua stuygp iso ‘x no 
alle[OgIe JUDUISAS 
(11) lejogse JUSUIS (Fd - 1)!u'us Yd 
ouIQU ne JUQUUSTIEdde 7 19 1 IS ae 7 
=" 
SIUDIQIJIP SoItepogIe s]UIUIZES ee ft 


sop B juguuarpedde 7 39 7 Is ae 


"INS OUIUIOD JouINsa asstey as “4 ‘aguiqui0s 
UOISNOUL,.p ay1]1qeqold ey anb JaNUOUI ap a{qIssod jsa |] ‘UoU 
NO QUUOTaIIS }so JURPUOdSaLIOD JUdSWIZaS Ina] aonb uoTes 
‘UOT[HURYI | SUBP SISN[SUI UOU NO SasnyoUI xNap $a}no} JUOS 
1] 38 1 SayluN soy ‘sed [9] UN SURG “aITe[OSIe aSeq Op JUSUISOS 
OUIQUI Ne jUUUSTIedde 7 19 1 SoyIUN sa] onb suosoddngs 
“y'w e@ afed9 juowlosessoogu sed ysa,u “x ‘stogoino} ‘ore] 
-O9Ie aseq ap UOT[MURYSE UN InjouT UO NbsI0_T */2'u eae 
1S9 ‘UO][ULYIE,| SUBP SasNOUI JUdIOS 7 19 1 SgyIUN sa] aonb 
guyiqeqoad ey “1 onbsioy yuowonbrun sapryea yuos uoTeUI 
-1]$9 UOS 39 (9) Ud AY INod soURLIvA ap d[NULIOJ VT “quUSISsaYy 
DOLNeU Vl Op ISIOAUI,| ISA “{ 10 (,) Ua TULJgp Is0 V no 


(on “VV + 


1='W 


4 = (4 


fa hat, oe ta! ou 


Jed aguuop isd AV op 

SOURLIVA BP] OP UOTeWINSa ouUN ‘(//6[) UBIYDOD side a 
‘ajyuepuod 
-9puI ovjogse aseq oun yo “"g* ‘'g ‘saysty y JUeod 
-W09 SBd Ne dIPUd}9 JUSWII[IOvJ JSsIe] 9S INDJBUINSS 19D 


! 4, wornueyos 51 

(6) = i Oa 

so uoNe[ndod vy] ap ate} ey nod guise uosdwioy [-ZAIOY 
op es uQ “‘JUaWIaATIOadsal ‘(¢) Ua 19 (py) ua seladen 
qUOS © "qd 19 ‘uw soqpiqeqoid soy *('u - 1)’d +'u = ‘yu ogou 
4s auiqeqoad d}9D “ailejogie aseq ek] suep No 9jST] oun 
SUIOW NB SURP SNOUT yOS 7 NPIAIpul un.nb sawiNso sy] Iq 
-eqold e] suO[Nd]ed snou Jo g B JUSWIIATILIAI (¢) B[JOUUOT 
-IpUOd SOUL[QUIASIVIA B] SUOSTUNTXeUL snou ‘aynsug "Q/"n 
="d anuuoo gnuenb ey ayuasaidar yo soqtun seq saynoy 
Inod dUIQUI PB] ISA DIIe[OIe aseq BI ap UOT[MURYDY,| SUP 
UOISNISUL,p si][Iqeqoid ey] “sagarlasqo JOS SsQUUOT[HURYIg 
SJUSWUIZAs Sop Sa}1UN SaT So}n0} anb suosoddns snoy ‘guUON 
-d9[9S 1s9 sjusWIZes “n ap a[duuts s1oyegTe UOT[HULYDE UQ 
‘g1911U9 UOTe[Ndod ev] JUSIANOD Inb syusuIZas "7 op en) 
-IsUOd 489 aMIeOgue aseq ev] ““g 19 'g sayst] xnop sap sn{d 
Ud aIIe[OsIe aseq oUN B Sgdde SUOAR snou anb suosoddng 


SIATV[OIAV SISEQ IP 39 Sa}SI| Op 
opie | ¢ uonejndod ef ap ay[le} ej ap UOHEMIS| 77 


“UOIJL[NUIIS LI Op s}eI[NSaI Say ¢ 
UOT}9AS PI B PIBANOI) UC “SUQsOIDI9Y JUOS UOISNOUT,p Sat] 
-Iqeqoid sat anbsio] gudoidde sed js9,u ‘stoyaynoy ‘Inayew 
~$9 J9D “‘SoOUgSOWOY UOISN[OUr Pp sayt{Iqeqod sap yueyOduIO0D 
ouNoeYO ‘sayst] ¥ B UOTRUINSA,p saInpgooid say puss (661) 
SOUIe} “SOUQSOWOY 1UOS UOIsSNIOUI.p sait{Iqeqoid say anb 
32 sojuepuodgpur sajst] xnop v X [I .nbsioy Ay uonendod ey] 
OP J [le1 LP] Bp s[[OUUONIPUOSUT soUL;QUIASICIA Op UINUITXeUI 
NP INJCUIT}sa | IS9 UasIa}ag- RUE op INoeUII}sa TJ 
‘quowsanoadsar “¢g 19 'g saysi xne erates 
jueuoyedde soy1un .p aiquiou of juoustsap “a 19 “Ay no 


ysa "97 ap onbnoduidse adA}-ineu9 | “(6p6]) SunUaq 
19 Jeyag saide.q ‘([S6] ueuldeyD) udsiolag-ujoourT 
ep InoyeuNse, onb gsteiq sulow jso InoyeUINSa 109 


Clg 


(8) Hee aes 
(1+ ®N)(1+ 4N) 
Jed aguuop ‘uasiajag-ulooury] op Inojeul 
“89,] OP S9tJIPpoU UOISIOA ouN gsodoad b ([ 6) ueudeyD 
"[NU 10S me 'N iAnayeuruougp oj enb ynad as [] ‘sayuepuod 
-9PUI So\SI] XNap sap ouNdeYd Inod saugsouioY UOIsNoUI Pp 
sauiqeqoid sap juayodui09 saqtun saj sajno} onb juepnoa 
asoquiodAy,| Ins sapuoj sJUSUIOW Op UOTeUIT}Sa.p o[duIIS 
SPOYIOU! OUN Pp U8L,s [] “BsepUOS op Saseq Xnep xne souNuUI 
ee soi Pp aiquiou 2] auSisep “@'%A7 39 poh oa? 


“¢ ya 'g saist] sop ayer ey quousisap “7 yo 2A no 
¢q'q 
‘ N a ada 
va I 
In IN 


“JINS SUIUIOD ajUasgid 9s 13 (9681) 
uasialeg & 19 (OES) Ufooury & a[qenque so onbissejo 1n3} 
-PUINSS 192) ‘UdS19}9g-U[OUTT] Op INdJVUITISS | ISO IISTessal- 
Jistes od} ap saduatiedxe sop suep JUSUTUTeINOD 9gsITNN 
so nb uonendod ey] ap a[Ie e] op InsjeUINSe one UA, 

(9) u 
zed (9)'u op uoneUNsa | B s[QeNnque 3So (1) Ue sIqUISUI 
puodeas a] “OUUdISSOY SOLNLU! LI OP SSIOAUL,| 1S9 (o)< re) 


pe Ot | tw 
- =V 
(gue ! ; 
(L) no 
2 (9), 1='W re 
“K@xr+— =(N)A 
teeth (0) eP1 se 


uoneindod ap sasyyiyd sap je alle} eB] ap UONEWUNSA -eiNUed }2 YOO||Od ‘Seule}] Bel 


SULIOJ P| 
snos ajuasaid 9s 39 JO[AR], 9p Spoyyout LI op apre | e anua}qgo 
189 \V ap soueLIeA e ap UoNEUINSe aun) ‘(p) ua atUlyap sa 

‘yw Wo ‘'u/] i We N sed aguuop uonejndod ey ap aritea 
BI OP UONLUINS,| SUOepIsUOD snouU ‘nuUODUT Iso @ enbsing 


a=W 


. ¢ as 
yop we WA 


1SO AY Op doURLIRA P| Op UONeUITSS OU- 


‘yt ee 
(9) ok NA 
eae es 
1S A’ Op GOURLIRA P| ‘(LL61) ueIys0D saide.q 


(7S61 uosdwoyy, yo ZWAIOR) ‘usp '* Gee N Welos NV 
op uosduloy-ZiAJOY op Inoyewnsa,| ‘nuuos 119 @ IS 


(s) toe wane Pe 


ng "p) dxe 
eG ig ) + | hig 


v 


(x ‘g + “p)dxo 


: (xg 4 “@y) dxa + I 
(y) iO) tbe Fee Spe ea Ce 


feat - 19) dxo + I | 
I 


* 99]0U 1S9 SajSI] Sop UN SUIOUL Ne SUBp SLIdUIOD }10S 7 NPIAIP 
-uI,| onb sauinsa sujiqeqoid ve] ‘uoIMEN op Spouse P| ANs 
s[Ieiap Sop (L66[) SouleE] suep Y oxouUR | B BIDANON UG 

(2g “4p “8g “%p) = g sqqou ‘g anod ayauuoNIpuos aour]q 
-WOSIRIA Op WINUIIXeU Np sINoyeUINsd sop Jo[No[ed Inod 
UOIMON Op opoujsur eT suonbrydde snou jo (¢) JastwuIxeW 
inod sauesyyns sanbysyeys so] Ins agpuoj BARI oINp 
-9001d oun gsijnn & (0661) OUTY *,( ‘ag “Ey Ag <p) — g Y9 


(ix(“4@g 2 tg) + 8p 4 hy } dxo + 


(xg +} dxa + ('x'8g + %n} dxa = (0)'y 


@)'y "uy 
(a eC: as es EK lg + ‘any ‘dy 5 ‘day ‘hy }dxo 


:1INS OUIWIOD $ajsI] xnep nod a][auuONIpuoS 
SOUL[UIDSIVIA OP UOTIDUOJ eI QUaSQId & (Q66I) OUTV 


(yet 


ZEl 


"JUDWOIINe () 19 BSePUOS op soseq xnop 
sop oun suTOW ne sup sUdUIOD }SO 2 NPIAIPUI.| IS [ 389 


z l 
(Z) 'p + an in ='W 


UOISSOIdXa | OP INI[VA eT 


“‘yuowWoTnNe CO 


a8epuos ap saseq ='D 
XNop So] suep SLIdWIOD }S9 7 NPIAIPUL,] =| 
ey a & f 
re) 
juswiayne 
juswlatnes / asepuos = "n 
op aseq RB] SueP SLIdWIOD Jsa 7 NPIAIPUL| =| 
iN beer aT 2s 1 


Inod snossap-19 SddLNVOIpUl soTqeLIeA So] SUOSSTUIJSp 
‘Q{OUTISIP dJVIIS SUN SUIWOD NPIAIpUl onbeyo queyweN uq 
‘soquep 
-uodgpul sajst] sto) Inod a][auUONIPUOS sdUR[QUIOSIVIA Op 
UOTIOUOJ LP] QALIOP & (L66[) SOUTRH{ “SUIOU Nv o)sI] 9UN SULp 
SANOIOI 9s gy1uN | onb e[[euuonIpuod SOUR[QUIDSIRIA PB] Op 
UOTJDUOJ Ud SnuUd}qo JUOS ‘¢g 30 “9 op courjquiastesa op uINUI 
-IXBUl Np sInajeUINss say “x e a]JoUUONIOdo:d agsoddns 
iso “d no ‘(add) ajjrey ey e ajjauuonsodold ayt[IQeqoid daAe 
SeUUO][NULYDE,| OP DUSIOJJIP Iso UONesINOWIeIed 309 
“Q[QUIIVAOD PI OP UOTOUOJ UD DJUSWIBNe UOISNIOUI,p sII[Iq 
-eqold eI ‘QO <‘g Inog as 10 ‘n ap SINE|VA Sez $9]N0} Inod 
| >'d> 0 ouuop uonestnguesed ay90 onb Jaj0u y ‘snuuod 
-ul sanjguiesed sop quos ‘ 39 "py 39 a[qeUeAod aun ys 'x NO 


(xg + "0)dxo + ee 


(1) vce aides 1 
(x'g + 0) dxo 

Jed saguuop ‘uotsnjout .p sayyiqeqoid say 
Inod onbnystZo0] sjepour a] gidope yuo sinayne sa] ‘sasodxa 
XNap S9d sue] ‘dISIeSSdI B[Nas oUN 39 JISIes OUN INS jUep 
-UOJ 9S doULI} UONR[Ndod oun ,p a[Ie} ke] Op UOTRUIN]SO,| Op 
QNA Ud I[[SUUONIPUOD UOT}eUIT}SO Pp sINpgd01d oun gsodoid 
1U0 (0661) OUTY 1° (6861) SUIssN] ‘soyUepuodgpur soyst] 
ap siquiou Z| ‘¥ = J & SaTqe{quuas yUOs aBvas—aId ap sath 
1°" ‘T = £ say ‘stoy oun aloougq ‘aseaseid ap sajoAo y ‘T= £ 
yo SNpIAIput Ay“ ‘| = 2 JUoWOdUIOD saguLIay suone[ndod sap 
Inod i1stessoi-istes odA} ap saoustigdxa say] *(uOIsN[oUT) 
aIstes op sayesaul saiyiqeqoid daAe _ IISIessaJ-JIstes 
od} ap saouatiedxa sap suep sollelixne solqeiiea sop 
9[QJ 9] QUILIEXO IUO (0661) OUTY 19 (6861) SUISBNH “(/861) 
S[OYDIN 32 SOUTH “YOOT[O”_ “OelIxne sjqeuea oun ,.p UONdUOJ 
Ud UOISNIOUL.p sayiqeqold sa] Jasi[gpowl sp 3sa Jopgo0id 
op uoses one oun ‘anbinguiesed uou o1sgiens ey ued y 


000g eiquwesap ‘ajenbua,p senbiuyoa | 


‘S[QPOU! OUIQI-y 
g[ Inod uonejndod x ap are e] op uoneumsa,p onbiy 
-guiesed uou spoyigut oun gsodoid yuo ‘uoj[nueYyo9,| ap 
QINSANOS ap V9pt_.| JUeAOTdUID ‘(766 1) SUay 19 dar] ‘ovYyD 
"SajSI] Op aIquiou gf ‘y= 7 NO ‘sanguiered [ + Avy eV nbsnf 
Joyoduoo ynod [1 ,nbsind sowinso e aproey sed yuowuraptas 
1S9,U 9[QPOW SUIQI-Wy YT “aIst] SUIgI-/e] ap JUSWE[9 OUIgI-1 
Np WoIsnfour.p ayyiqeqoid ey suu09 “d suossiuljap snoN 
“NPIAIPUI | UoTas onb (9}sT] op SULIOJ snos aseq) advasgid ap 
Q[DA9 9] UOTSS JUL} JOA JURANOd UOISNIoUI Pp sait{Iqeqoid 
Sop JOWpe I[Qpoul sUWIII-~y Np sULIOJ snos gutIdxs 
Q[QPOUW JJ ‘IIsTessaJ-IIstes adj ap saouatgdxa sap sueq 
"JUBSTeJsyes Jey B No} sed jso,u Inb onbin 
-9uesed UOU soUR[QUIOSIeIA Op UINWIXeW Np InayeuNsa 
un gsodoid quo (9661) YOOT[Od 19 SHON “9iI9UESOIDIDY | 
JOSTUIUIW & UOSeJ Op B[QUIASUD UOS sueP OpNId,| SATOSUOD 
uo ,| onb gig33ns ouop IO (8/6) ‘110919 SO “Ww sIEepowl 
Np sped os] suep AY op jULSILJsIVeS WeJ B INO} IndJeWINsa 
UN JOANOI} Op I[LOLJIp 389 [I ‘[eIgues u_ “wWeyUWing op sjIuy 
-yoef inayewnso | aonb gsteiq sutoul so ovyD sed gsodoid 
INIIJCUIT}SS [| ‘S9OULJSUOIIIO SOUTLLIZO SUR “([66I YOOT[Od) 
aistes op souonbyj ve] Ins saguUOp ap UOT}DUOJ Ud s[apoul 
99 Inod JUSWOW sp INd}eWINsa syne UN s10QPIP B (8861) 
ORYD “(961 A[fnouend) steiq np UoKoNpgs op ayruyyou! 
Opoyjoul ve] Ins opuoj as (ZL6[) WeyuIng wWosp onb 
N inod gsodoid inayeunsa J “Istessoi-sistes odAq ap 93x9]U09 
un suep AN op UoneWsa | ap owiefqoid 9] suIWexXa jUO 
(6L6I “8L61) YOUSAG Jo WeYyUINg I (7L6]) Weyuing 
"ql “soastt ¥ sap ajquiasua | Jnod a1uB)su0D 
yS9 1 yUOUIZTE,| Inod ‘d uorsnjour,p sipiqeqoid ey ajjanbey 
juRAIns ssqujodAy | @ puodsaod eja0 ‘saydy[nwi sayst] sap 
Inog ‘9iUN sUWIgI-7 e[ INod UOIsN{oUL,.p dyIqeqod ev] aUSISEp 
‘d ya uoneindod ey] op aye eT so AY NO ‘NS ST =a1‘'d 
19 AN OlIp-e-1so,9 ‘Te}0) ne sanguered [+A e,nb 
-snf 1oy0dw09 ned gylguasola}9y ,.p sJapoul a] ‘asvas—eid 
OP SaJDA9 Soy sno} Inod sauigui saz segsoddns juos ‘ane 
B yewiue uN.p JoueA jueAnod onb ualq ‘arses ap sai[Iq 
-eqold say ‘Istessal-listes adAy ap 91Xa}U09 uN sue “(8/61 
UOSIOPUY 19 DIY AA ‘WeYyUWINg ‘syQ) seuey UoNR[ndod oun 
Inod Jistessal-1istes adA} ap soyosewep Say Ins uoNeUsUINS 
-Op BI SueP s119UEZOIDIDY ,P 9[Qpou 9] BUBISEp “yy eJepoul 
Y] ‘SIUSIOJJIP dIstes ap SafdAO SAP SUIWOD SogI9PISUOD aIIg 
WUDANAd Sa}UdIIJJIP SISI] SOC] “dISI] Op SULIOJ snos asepuos 
op aseq OUN SUeP UOISNOUT ,p say[iqeqoid sajusigjjIp 1ou0d 
-W109 JUSANAd S[ONPIAIPUI SJUDWIIE Sop ‘OUIQUI aq] “sISTeS ap 
soqiqeqoid sayussayjip juayoduioo xneumtue syuargyjip onb 
jnad as [I ‘Iistessal-1istes adAj ap saoustigdxo sop sued 


S9}St] 9p 
apie | & uoyendod ey ap atv} ej ap UONVUInS| = [°Z 


SANOILSVGADSOUALAH 
NOISNTONI.G SALITIAVAOUNd “7 


"OITBUIULUOS 
UN J9 SUOISN[IUOD sap jUDUSSaId sINayNe soy ‘UIJU ‘uONR] 
-ndod ap saijiyd sap 1a (uonetndod) aseq x] ap oyyieI P| 
OP SINSVUINS SIOAIP JoreduIOD ap sted e mnb uonernuts 
OP EpNi9 9UN_.P S}eI[NsoI So] OUINSSI ¢ UOTIOAS eT “aITeTOSIe 
aseq OUN.P 39 Sa}si] SInotsnjd no ouN.p sei sjuaWIOUSIES 
-Ual ap s0Uasgid Us UONe[Ndod ap saiyyIyo sap 1a UoNE[Nd 
-Od 8] 9p o][Ie} B] Op sInayewse sep jUasstuINo] Inb sapoul 
-9UI 9p UOLsonb jso A [] ‘sonbnsepegosolajay uorIsnpout ,p 
soufiqeqoid sop jusyodwi0s sjusulgte seq JUOp sajuepusd 
-9PUI Sd}SI] Sop JUOUTWIeXS sINoyNe sa] ‘Z UOT}OaS RT VW 
(S661) [930A 19 NOY suep [Ielap Us souTUIeXa JUOS sate] 
-OgJe Saseq op Ied ev aseuUOT[HUeYDI-sNos 9] 39 BseUUOTIN 
-UBYO9 ‘TJ ‘Sore[ogIe Saseq say Inod eseuuo][HULYDg ,p 39 UO} 
-ONIISUOD op soxafdu09 snd sapoyaW sap (9861) “]]09 10 
0S99,J SUBP PIDANOI) UGC ‘SalleTOgIe saseq sap Inod jusuUIas 
-1991d sn3u0d sorteyuourg|ddns sinoyeurnss sap a[nuoy U0 
(1661) tnorey 39 Auaquoyjne.y ‘saigpuod 3a SoULIoy ‘SLIOA 
-NO SJUDWIZAS Op sINdJeUINSa sop se|[IvJap UoNdiosep oun 
QUUOP B (P86) UOTVAN ‘sajusutTjJed seguuop soy] a[Iande1 
UO 19 QUUOSIAd Ud DIQUIOUEP Sa] UO ‘QUUOT[NUPYSE JUSWISaS 
onbeypo ap ules ne sagijUSpI 939 1U0 ‘saJURIE[OEp SayUN So] 
No ‘saToolse suOTIeO[dxo sa] onbsso7T ‘arejoare aseq ey] op 
S]UDUIZ—aS So] 39 UOTe[Ndod ey] ap sayooLse suoNeO]dxa soy 
aUd SUdT] SIT JWassI[qeje Inb sa[saI Sap WULJEp UC (S661 
[230A 19 NOY) sayNeNs soisop sinaisnyd ve asepuos op suejd 
ap opie | & SQUUOTOIIaS SIOTe JOS ‘aseUUOT[NULYDSE ,p soyUN 
Sa] 1UOs Inb ‘s}uaUIZas Sa] ‘sJUaWIZas sop sogfodde sanbiyd 
-813093 SUOIS9I SP SLIIS SUN Ud S9SIAIP 3S9 IQIIJUI ,_P UOIS9I 
RI ‘ajoouse osjgnbus oun suep ‘Isuly “olle[oose oseq oun 
JULNINSUOD UILLIO) OP SUOISIAIPGNs xne sogidosse JUSWIII[9I 
-Meu JOS sajooLse suoneyoldxe say suwW0d suonendod 
saq ‘a1qtd uonendod ve] ap aje[dwi09 aInySANOd suUN ITUe} 
-qO Pp UlJe SalIepOgIe Saseq Sap JUSANOS JUaST[N aJgnbua ,_p 
S9BILYO SIT “SO[GeIJUAPT soy] Sop Jed soruryop sonbryd 
-P13098 SUOIS9I SP UOTJIO[[OD oUN ISO SITe[OSIE aseq BUA) 
‘uoryetndod 
ap sajiyo sap ya uone[ndod ex op a|[lei e] ap saovotjyjo 
SUONPUINSO Sap II[qeI9 o119-Inoed elinod uo ‘aitejogie aseq 
9UN .p 91 UOT[HULYIE UN.p opIe,| & so[qruodsip sjusursUs 
-I9Sual Sot JURJe[duI0D U_ ‘UuoNR[Ndod eI ap aT]Ie} eB] Jase 
-snos ap juanbsi sajst] sap Ins yuouonbrun juapuoy as tnb 
SUONRUINSA Soy ‘sajgtduoour s]e19U9S 9[sQI UA JUOS Sajsl] So] 
anbsing ‘juewa[qnopep ep enuuosut wed oun juspodwi09 
No ‘sajqist]I ansed us 32 so}q]duoour JUsANos JUOs anbuod 
-jonb yng un inod juawerfeniigey w]qQeI9 UO] aNb sayst] 
sa] ‘ogIpme uonrndod xy] eg yuaura}9ex9 puodsaiioo Inb 
dIST] DUN ITUDIGO ,P IILOIJJIP JUSANOS 3S9 II “(LL6]) URIYIOD 
saide .q *soje[dwooul sajst] XNep ap SagJI] SUONPULOJUL,p 
UOTOUOJ Us BBEpuUOs ap Saseq Sap ay{Ie} BP] op UONeUINSS | 
ap ona Ud IIstessal-Iistes odd} ap sgst[gpow UoNNTOs oun 


uonejndod ap sauyiyo sep je alle} B| aP UONeWIISY :ejnjUe_ Je 4IO||Od “SSUuleH 9€l 


juoupedag ‘AjIsIDAlU:) B181g eUTTOILD YYON ‘epnjueg “5 Anses 39 YOoT[Od "H youusey ‘EEZOT OC ‘UoISuIYseM ‘snsuaD dy} JO NeaINg ‘¢ ‘— ‘souTeH “"g UME 


giuasaid jUO (p66) UMOIG 19 JOWIN], “YOoT]Og ‘slup-sieyq 
sop anynousy jo juounredaq 2 Jed sgsi[yn juos In 
Sollepogie Jo saydnjnu asepuos ap saseq sop kB Sal] SINS}eU 
-1]S9 Sd] ANS S[IeJEp Sop IWINO] & (PRG) UOTVaN onb stpur} 
‘SIU(]-SIEIG SOP B[OOLISe Ind}9as 9] Inod sagsi[NNn adepuos op 
saseq So] anaal ua assed qUO (96) [230A 19 BION, ‘0sda,] 
“sqjoouise sonbuysiajoeled sane .p 39 sajoolise sosuodep 
SaJ ‘UONONpoId ap s}NOd sa] ‘SaTe9I99 OP SYIO}s So] 19 UOT] 
-onpodd eR] ‘[reyaq a] ‘sagousWasud saroljiodns soy Ins sou 
-UOP SOP UINSAI 9 JI[IONIII SSYWN AT ‘Safoouse syinpoid 
XNOIQUIOU dp UONLUINSA ,p 19 aseUUOT[NULYSE ,p SoudIeUISp 
sas Inod a8epuos ap saseq sinaisnyd JuouaT[enjoe ssI[IN 
(201A4ag SOISYDIG JOANJINIASp JOUOION) SSWN IT 
(9861) 
dIRIAT 19 UBMOD AP 12 (Q66I ‘9861) JOIOM Op Sesodxe 
So] puaiduros inb ‘uauasuade1 np JUSWIsIqUIOUgp-snos 
ap swig {qoid np juswaratnorped snd juezeI) IIstessol-IIstes 
add} op sayoewap so] Ins uoTyeJUOWINDOP ke] op og}0UUR 
a1ydeiszoljqiq sun gjuasoid & (766) BIOQUOLY ‘soUgZ019} 
-9Y UOISNIOUI.p saitiIqeqoid sap seo a] guruexe sed yuo .U 
sgsodxd xnep sag ‘asepuos ap saseq sinaisnid ap uosieu 
-IQUIOD P| ap Isa INb 99 Inod syfeiguas snd anbi09y} o139) 
-eI]S UN g]UASEId JUO (G8GGI) YOOT[O_ 19 Sourey onb sipuey 
‘gyour[q 939) & sonsiesid ap uone[ndod oun ev asepuos ap 
soseq xnhop sop aspoyjaw ve] anbrydde yuo (eg66 1) YOoI[Od 12 
souleH (9L61) pio, 39 Joysesog & 19 (761) JoIstouling 19 
JOIN & (961) puny v “(S96T) UeIYDOD ¥& snp JUOs sonbiI 
-09U} sjusWIoddojangap sane qj “asIejOgse aseq oun 39 aja {d 
-WOODUI 9}SI] DUN SUTQWIOD “(p16 ‘Z961) AoTeH Jed proge .p 
slOgRiEe ‘oIsaeNS DUE, ‘dINOIJJIp 97100 Ja}UOWINS Inod 
UOTIN[OS ap sUOTIsOdod sasIOAIP JINjeII}II] BE] SUBP SAND] 
uo “(€S6] MOPRIJ] 19 ZIMINH ‘UosuUeH) S}OeXOUI s}USWIDT] 
-SIZOIUD SOP 19 SJUSWI[GnOpep Sap ‘SUOISSTWIO Sap SUTUIOD 


"WSN ‘€078-S69LZ ON ‘YSIEIRY ‘E07 XOg ‘sonsneIg Jo 
if 


SUOTIIJIOdUIT Sap JUSWIITQeITAQUI JUSTIOdUIOD BBepuUOs op 
Saseq Sd] ‘S9QUUOP ap 9}92][09 op uONeIado ajueyOdUIT 93n0} 
suep ‘anbsing ‘snuuooul uonerndod ap saijjryo op 39 aTIIe} 
OP SUOTJLUITISA SOP ITUINOJ B JUDAIS SaIIe[OgIe saseq Sd] 30 
SO}SI] SY] “S]USUUASSITQeI9 SATIN ._P NO sasudaqyue sop ‘xneul 
-Iue Sap ‘souUosiad sap JdIJIUSPI & JUSAIOS SASIOAIP UOTIRIIY 
-1]21]S OP SITQUURA SaINe Pp 39 SI[QeLIVA SD “JUSWUD.e[dUIS /p 
SOT[OU9ILUL SUONALIOSap Sap NO g[RIDOs 9yLINDgs Op sOJSUINU 
sop ‘auoydg]a} op sorsuiInu sap ‘sossoipe sop ‘suIOU sop 
‘saune oud ‘aipueiduios juaAned aanoy A uo nb syuawisye 
SY] ‘a[qQIo uoNe[ndod ev] sp seseuUOT[HURYDE.p soyUN sap 
juaiguINus Inb asepuos ap saseq sap sajst] sed puajyud UD 
‘g09ds9 | No dIIIe} eB] ‘axas 
J] ‘ae, UOTAS JOLIeA JUSANEd xneUTTUe say Inod (UOTSNOUT) 
aisles ap soiypiqeqoid sa] “xneurtue so] Inod sayesoul yUSIOS 
daisies ap sayyiqeqoid say aonb ynad as [I ‘IIstessad-iistes odAy 
ap saouatiadxa sap sueq “sayed snjd suoneit0[dxe sap aonb 
OBEPUOS Op VSeq OP JULAIOS 9)SI] JUN SULP sasNjdUI oNg,p 
SOOULYD SdINIT[IOW op jUSTe suOTye}TO[dxe sopueis op onb 
nad 9s [J “sourssiod suonezo[dxe sap ayie] eB] op UOTeOIp 
-UI oun yUOS saXolduia .p sIquIOU 9] NO souTdIod suoTje}0]d 
-X9 Sop o1dI}Jodns ve] SUILIOD d[QeLIeAOD op SoINsoU soq 
‘PION Np ouljoreD usa suOYdOd op sIquIOU 2] 39 SoUToIOd 
suOneO[dxa Pp SIQUIOU d] IOWINSS,p assIdv,s [I,nb suosod 
-dns ‘tsury ‘a[qeueaod oun,p oipuedep juoAned sougsol 
-919Y UOISNIOUI ,p saz[Iqeqold sa_ “asepuos ap saseq SOSIOA 
-Ip Inod uorsnjout,p sayiqeqoid sayusrajjip yuoyoduios 
uonendod ey] ap sjuaulg[a say onb gsoddns 4sa [J ‘soyuepued 
-9PpUI asepuoS ap saseq sInaisn{d ap soa) suoNeULIOJ 
-UIL,p aouasaid us uoNendod ap saajjryo sa] 39 UONe[ndod e| 
OP a]Ie} VP] JUDUITISA sInayne say ‘asodxa yuasaid a] sued 


NOILONGOULNI ‘T 


“IustessoJ-listes odA} 


op sseUUOT[HURYOg ‘orIe[OgIe BBEPUOS ap aseq ‘a)SI] 9p SWLOJ snos asepuos ap aseq ‘onbusisoj uoIssalsay  :SATO SLOW 


“squgsOl9}9Y UOISNOUL,p sait[iqeqoid sap Jastjgpour nod onbrysiZo0] 
UOISS3IB9I BP I139}eIS EB] & SINODII JURA UD UONLINdod ap saljJIYyO Sap Jo BBePUOS ap Sase Sap II][Ie} E] Op SINJBUINSS SIOAIP 
Joredui09 ap jouuiod inb uonejnuits ap opm oun _p syei[NseI soy JUo}Uased sinajne sx] ‘onbysIZo] uoIsseJZaI op a[Qpour uN e 
jodde juesrey 11stessoi-iistes odA} ap sopnye sop suep Salrerprxne sotqeueA op UONest[spoul e] Inod (Q66[) OUTW Jed 19 (6861) 
sulssny Jed agXojduwio apoyjeul ek] JSTJIpOUr Jo yUSIdepeE S|] “SoTQeLIEAOD IP UOTOUOJ Ud SogSI[SPOUL 9 SAUQSOIOI9Y JUOS 
$9}SI[ SO] SUBP UOISNIOUI Pp sazt[Iqeqold sa] NO sed J] JUSUTWeXO SINojNe sa] ‘UONL[Ndod sp sarjjtyo sap 39 d]]Ie} VP] Op saoKVdIyyo 
SUONLUIT}SO SOP ITUINOJ B UOSLJ IP SAQUIQUIOD JUOS SajsI] OP Jo SdIIe[OQIe BBEPUOS JP Saseq IP Sd9JI] SUONEULIOJUI SOT 


aWnsaa 


WININVd ‘D9 AULSVS 9 MOOTI1Od ‘H HLANNAM ‘SANIVH “A NMVO 


$9U930.19}9Y UOISNIIUL,p SoIIGquqoid IaAV 
$9}9[AWOdUI Sa}ST] Op $9.11) SUOT]WUBYII Sop 1nod 
uoyelndod ap sarjjiys Sap jo o]]Iv) Bl ap UONLUInS| 


Sel 


epeuey enbysnejs 
vri-sel dd ‘Z .U ‘9Z “1OA 
000z aiquiadap ‘ajeanbus,p sanbiuyoo | 


GUSTY 
aljsx]q inj onb jouuondsoxa ang joo sIpuaiduioo xnotw 
ap sewed snou juassind jua}[nseI ua Inb xneaen soy 
onb juouryad sed |I-j1eJas aN “apeotpaul ayosayoal ey Be sdi09 
UOS ILIJO B 9ISISUOD BIN d}ISO19USB Op 9)0e JOIUIAap UOS 
‘JUI9]9 1S9,S [I ‘UOWOSNsINIYTR|Y “eSOYyo aTNe JaUIZeUII ,p 
9[qIssodunt sugUI Wesstesed [I Ie} US steU ‘oITeUD]UO 9] DIP 
-UI9}¥ IIOA 9] Bp B[qGIssod JUsWaTNes UOU We] quias [I,Nb aA 
ap uls{d ts y1e39 [J “sed sreyuesteyd ou of ¥q “JaIqg[go op WeUsA 
dIQW-Oulay ev] aonb inja0 ‘al[sa] op ailesJaATUUR IIQUILIDA 
ne jusssid a9 Joladse juesip ua gurus) re,f ‘aourssteu 
ap slesIoAIUUL ,06 UOS eB aored eB] std Ivf onbsio7 

‘squuosiad ap sJal]]Iu ap 3139-Jnad Ja sourejUua. op 
sinof{no} op we | 38 xnenjoajjJe asad a] 19 xnod9 | ‘[2ON ep 
S9I}9] S2T[9q sntd sap Inajne | ‘ajqenesul In9}de] 9] “INd}U0D 
9] ‘TI[IOIA stewefl eine .u inb surewiny s}101p sap inasuajap 
3{ ‘enbysnejs Jsa Inb 99 ynoi ap sydosoyjryd 9g] ‘ausedsq .p 
SIAIO BLOND ke] JURpudd juReQUIOD J :«aINIeU BI Op 9d10J 
OUN ‘YSTy SI[saJ» suguiouayd 9g] asstey snou [I ‘anbysiyeis 
R[ Op oUleWIOp a suep anseaz snou [I,nb 90 aNNnO 

‘(AW 
NP YZPIO,| IP JBIDIJJO .P XIOID LJ 19 DLIBUOP PI op saouatos 
Sop SIWISpeoyY ,| Op ole1OUOY sIQqUIOUI ap ann ‘\sodepng op 
S'USIOAIUN opuLis snid ey] op anbiylouoy ye10}O0p) sUIsUOH{ 
RI Jed sagigJuod SUONOUNSIp sainey op spnyn{Nu ey Jaydu09 
sues ‘xnoX sas B SATJBOIJIUSIS snid e] soURSSsIeUUODAI P| Og 
-jnod 39 onbysneys ve] ap [eqON xXtid op suIOJ ouN oUTWOD 
aigptsuod of anb s]j,] ap sle1ouoY sIe[NI] sIquIOW op 
211 9 ‘URSIYOIPAY Np sUSIOAIUA J op UOTOUNSIp sIney snd 
PR] WOs ‘TTassny ATUSH aleyd ke] ‘WSV,,| 9p UONOUNSIp siney 
snd ve] 10s ‘syIIAA [ONWIeS a[[Iepeu eT ‘UOWssI[qeI9,| Op 
DIVSIDATUUL ,006 NP UOISeI9O | B DUSO[Og 9p SIISIOAIUL) | 
op onbiyouoy 3e10190p UN JOUIID9P NA S98 |] “S99ILIQUI 


JUBWANOAEP ap BIA BUN - YsIy at/se7] :16a1/94 


1S9,S [I,nb suonounsip ap soureznop sap soun-sanbjenb 
JOUSI[NOS op [eJO}UD}UOD OUI oF ‘a[eUONRUIA\UI JT[OYOI | 
RB OURSSIBUUODAI SUN NPA jUO IN] suONesITesI sag 
“QIpUusjua ,p 
guuop 919 BW [I.nb YSV.] op JUapisgid np ajuenbiew snyjd 
RI UONNIOTIL | “(QL6] SW US PSP ZI Suep ned ve ajjenbey) 
LL61 YU VSV.I Op iuapisaid ap ann e& UOTINDOT[e UOS 39 
sonbrysneys SopousU Sap JIeI[NSUOd 9}IWOD a0U ap Ind} 
-BpUOJ SIQUIOUI op oyI[enb us epeuerD onbnsueis & SddIAIas 
xneAO] Op soguue sas :INd0d B JUSWOJET[NOIWIed yusUUNT) 
ou Inb suOTINqLNUOD SasNdIqUIOU $39 ap xNap JdUdTIS snos 
Jassed sed xnoaa ou of 1q “USEy Osa] op sjueIpnig sap suUOp 
JUSWIOU UN eB 939 sed jUO ,U snUUOD UaIg sseUUOT[HURYDE | op 
slasuri9 saysi[etoads ap uatquiod apuewiap ow af :sjuasoid 
-IUUUO JUSUIS| IDI] JUOS s|I ‘presey Jed xno ins snjd «aquioy» 
ou uo ‘Iny ,pino{ny ‘sduia) np ste jo sjuvIpNg susIOUR 
SdS INS «IOQUIO}» OP JALIL UW [I ‘SoTPUONRUIOIUI SamUOIUII 
Sd] SULC] ‘S[IOSUOD OP SODIAIOS BP XNBARI] 19 SADUDIQJUOD Op 
apmnynu oun sed onb tsure saruussep sep stndap sudanua 
uD1dysyojs Kaaing ayy uneat[ng np sasuodel ap jo suor 
-sanb ap sanbruosyo sep one | no oun | & UONeJOgeEI[OD op 
yo UONIPE,p [eae 9] Jed ‘a19,p sutuess01d 3] suep guUOp 
[t,nb sinoo say red sgytanoe sas yinsnod [I ‘ayrenel es std 
JUSUIST[SIOIJJO OAR saidy ‘siasueNe susIonsyeys soy Inod 
919,p suuesZ01d s1epuasg] Np a1ped 2] suep ‘siosueNe nb 
SUIBOLIDUIL JUR} ‘SUaTOTISIIE}S Op SUOTRIUNS Sap B OUSIASUA [I 
‘Joqiy WY B Yodvasay Aaaing 40f anjysuy | NUZASP Iso IND 
39 Op SINdJepuoJ Sap aiquiou ne Ja}dwi0s ap snjd ug ‘uoN 
-elupe ja joodsal a8vuUOT[HULYDE | B gJgJUO UO sonbrjdde 
Xneaed) SJotwald sa] yUOp souUOsIed sores Sap 9UN | 1NJ [] 
“SaTeJUSUepUO] Issne JNO} JUOS SUOTINGLUOS 
sone sas ap soureyiao onb owinsa,f ‘xneaed soo ap 90ue} 
-Jodunt | Ja1ogidgp IO[NOA sues ‘stey] ‘ossed ua [49 ‘suoIses 
soinad sa] JURSIA SUONRUITSo xne ‘INgQgep a] Sep “AaSsaIgJUI Ss 


VEL 


“‘OLO VIM ‘epeurp ‘(ouvjugd) BMENQO ‘S]BOD “H-Y 2IqQnouwwy] “vy uoT]das ‘a3v}9 auIT ‘epeurd onbusnes “yoyo Us USIONSHRIS “WaT[24 “qd uel 


It ‘sojdntnur synoofqo e suejd sap ydsou0d 9] s[NULIOJ [I 
‘g[oNUOD xtoyo np 1dadUOd 9g] SoU] [I {JUSWIOT[SANOUSI SOAR 
suO][NULYD9 19 SJUSUASUa.AI Sap o0dR | Ie} Os [I Sasuodar 
ap Inala. op opmja,] suep JoruuoId ap sJANoO Je [I 
‘(jayuely AURA] D9Av UONIOGR][OD Ud) Uo1HIWda4 pajbadad 
paoupnjog — agsuaduios uonnedas ouli9) 9] Jed mny .pinofne 
QUSISap InayeAoU ydaou0d a] JUIOd Ne JoW 39 saxo[dui09 
suOT[NULYD9 So] SUeP SOUdIOJUI,| Op UONsonb ke] Jo10[dxo 
B Siomuaid say ted 389 [I ‘sINOISIP 1}0U SUBP SOULS} SOO 
32 sjlidso sou suep osepuos op uvyd np sjajjo sap ydaou09 
3] WNponut ‘sayne,p seav stoyonbyonb jurypreaen ‘ysry 
al[sY]J “suoNdO]][e ,_p 39 Sasodxa.p ‘sapoe .p ssseul ouN .p 
sonjouod juos xneaed) sas ‘e[-a9uUe 33399 soide 39 JURAL 
‘sduro}-anuq ‘oJgnus sIA oun jueInp osinboe onbysneys 
soudtedxa oun sinjons 39 oisaqul Inb youwasay sof usisaq 
jooysyojs yered ‘(sue gf op oswuoy ounof un a109Ud js9 
UsTy st[sayJ onbsioy) gg6l u_ “suidung Aaaing sse1ANOo || 
op anbyeid assases op siosay xnalogid say Ins ‘ainsoul 
asre] oun suep ‘aindde,s jeipuow onbysnejis owiaysds 
3] ‘suas 390 uq ‘anbneid ev] op JueINOS [NNO UN sWWIOD 
Issne STeUl ‘(UOTeIIGSUI .P S9AdT9 VDINOS UN SUID ‘oTqIg 
Rl WP IeISUI.| & ‘dIIP-e-]S9,9) DUIRWOP Z| sUepP sIGIq oun 
UWS JUSUTNIs UOU dsoduiIs IND dOLBAOU DIANIO oun 
‘Buydupsg Aaaang ainsi sjonbsap aiquiou ne saqenbreural 
saseJAno sinaisnid gnse] eine snou [I ‘anbysyeiys uq 
‘JUDUIOUSTaSUd | B B1OeSUOD [I,nb 
sdui9} aj Ja}dur09 sues urayd sdura} e a3sod un yuedno00 ua 
NO} JVJOJOOP UN Jd VSLTLU SUN JUINQO [I :saquUe sorgTUoId 
soo jueinp onb gi[reaey jueyne sieuref eu jt.nb pie} 
snid elp [] ‘Youpasay jp1209 of ajnjysuy | op UONepUo] eI & 
odronsed [I ‘ajsersnoyjue odnois nad un dace UONeIOgRIIOS 
ud ‘NO JOqIY UUY & URSIYSIP, Np dISIOAIUE) | Op ssuel 
So] UlyUS jUIOl [I ‘/ pH] UA “SSULIE | SUP dITeJUO[OA SUWIOD 
Jose3ue,s Inod o1QWIed es NedAnoU op JdwoULU! I] 
“AINI[NIUBY ,| Op dQ sTUTW 
ne sind ‘juowWesuscel np neaing ne ploqgep ‘sratuuord ap 
sodnoiz sap & Ja1d0Ssv Ss ap 9ouLYO FJ & [I NO ‘UOISUIYSE AM B 
asvusUgp [J ‘sonbyewayyeUl Us suIQ Ip uN yUANNGO 39 YIOX 
MON OP SUSIOAIUL] .[ B SOPNIE Sas SUTUID} ‘sIUP)-s}eIq xne 
JUITAAI [I ‘GEG US BoUSedSsa oIIAIO aLIOND eI Op UL EI W 
“sinof sas ap a1SaJ 9] SUIQeY &,] 
sguitiddo sap 39 sked 39 ap Inoue uOg ‘ouSedsq UD saysIOsEy 
S93] 9UOD JIN] | B Jodionsed jo sayeuoneUsojUl Sapesiug Se] 
aiputol inod sapny9 sas ydu019}UI [J ‘INe[][IewW! opuou! un ,p 
uonesio eT & JonqiyUos Inod Jotjaoes as ev ygid neaAnou 
ep sioye Iso sue /Z Op SUIWIOY 399 sIey “a[DAO Jomueld 
np swigidip un s1ua}qo Inod sapnig,p ue un nb asi my ou 
IE ‘LE6I U_ “JIOS Np sInod Ud sapnieg sas yNsInod al[saT] 


‘Q[[TWUeJ VP] Op SUIOSaq xne ITUsAQns Inod 
JOT[ICACI] JUOIADP ‘sue QT OP 98" SIOTe oI[So] WwOp ‘sguIe 
xnap Se] anb aljrusis ejoo stews ‘alley ap JUassistoyo sjt,nb 
99 1S9.-D “SIUG)-SIeIq XN JUOIDISII S[I,S JOplogp JUSAIOP 
sjuryua anenb sof 12 al[so] op QUI eT] ‘apeo9p dI[sa] 
ap sed 9] ‘agate Ino saide nog ‘«onbuswy | Jouse3 
inod jueqnj1} ua sked np stos yUOS SUdAO}ID SOU op TWOP 
}0 UOT] UN» ‘Jaszor e[INY ‘sloIsUOY 9390d 31g9[99 9] IP 
B | sUWIOZ ‘sKed ingz UeINb sto1sUOH Sp SJot][IW op sourey 
-U90 Op IEISUL | B ‘STUP)-SIeIq XNV JOISIUD ,.p JUApIOep syuored 
sas ‘CZ6] UG ‘epuouU! np suOIZez seq sojnoi ap sudfoy9 
Sop Inoue | Jed VgAOUW SIA UN .p yUsNboTS sfoquiAs uN eI 
1S9,9 ‘syuarayjip sked burs e nusyedde e peidog ‘ai10jsTY | 
op sonboda sajusrajjip e ‘anb yWeyUOdeI [J ‘aInbeao[s us 
SONIS JULUDJUTCUI JS9 39 SIOISUOY-oONsNe sIdwI7q,| op sed 
sioye yestej nb ‘peidog & Q[6] Ud aU ISO YSTY Iso] 
“‘suonetndod ap aseuuo][HueYyse Ud UOTRULIOJ 
oun J1u9}qo e juowiaddojaasp ua sKed op sjuvipnye soy 
Japre .p ing oI SUP PUNY SMmo7ja7 JOUOYOUAIJU] YSTY 211897 
3] II[qeia & SanBa][oo 39 ste sas Jodsut .p 919 B OUISINTe /p 
$o}Oe SIOIUIOP sds ap UN] ‘o]QeSslIe}UI JUSUISNOAYP UOS 
NJ DUIUIOY 199 Op JHOUNSIP snjd 9] We. 9] ‘xNoA sow V 
“WestIg\eIe9 QI Inb a]qe[URIgouI o3eInod 
J] SOAR SOUICUIAS sINnaIsnyd syn] JOA saide ‘saitoyergdojsod 
suoneodui0s ap sayins sap WOU! 3S9 YSry ssa] “squue 
Jed stoj sinatsnjd spuoul a] suep ynoysed 1a8eAOA ap 30IN|d 
WeSSISE Ss [I {UOSTeUL BT B JoOR[dap as op ay1Oeded ev] JUOUIOTd 
-UIIS Sed }RIJIUSIS dU ‘SUR 6 B “SI[IQOW 99999 1q “a}QTdUIOD 
d]Iqow oun.p Inof yeyyey my [I soyoyorew ey] Inod 19} 
-do ,p inj Inod uonsonb sed 1139 ,u [I ‘npusjua ust g “Ja0e[dep 
as Inod a}ayoseW PT NO UOTIeIOdO J OUD IISTOYD WeAsp II 
‘UIDapEU UOS UOTas ‘anb sIJUOD PRAR WI Jo JIqns Weaop |[I,nb 
noues np ajorued onseydosyyre | op gpred year Ww I] ‘aNpol 
nod onbjanb 9y1{1qow! oun gI3;eUL ‘a9 39 JUSWIDLIOdWIOD UOS 
OP SoNddRJ Sa] $9]N0} suep assouNof es ap ‘WIeJ UD ‘Ja QyOROId 
-siod es op ‘auistureuAp UOS ap ddJ0J eI ‘JosnUIe Ss 9109 
WUC [I UITQUIOS 19}e]SUOD Op Slidins NedAnou 9p sIe19,¢ 
‘aidoid jrey9 Inq nb nuuod uatg yUsWAT[NAd 99 ‘xnaK say] suep 
‘SOAR INDO UOQ OP LI WAR [J “QUST[NOS 3199 ,P WeILIIUI 0729 
aonb enbsreusal sioye Stear ,f “odTeUDJUID UOS Jd1g9]90 op outed 
B HCUIA dIQP-dulay L] ‘1N0} saidy “oesIOATUUL 06 UN.P 
SbO Op jue} WestIey UO lonbinod juejuestejd us spuewp s10[e 
SIVA f “DOURSSILU OP SIIVSIOAIUUL ,06 UOS dp UOISed90 || 
e onbysuowny ned anbyjanb uonnoojye synod ouN gSIpeI 
te f ‘stout sanbranb aysnf no} e A [J “YShy SI[So] op atOUSU 
Rl & gpore un oil109 Inej ow [I,nb oio010 xnod ou of 


NOILONGOUINI T 


OAT “d NVAI 


JUIWIINOADP Ip JIA 9UL] — YSTY 9ST 


cel 


epeuey enbijsijeys 
vel-eet ‘dd ‘Z ,u ‘92 0A 
000z aiquiadap ‘ajenbua,p sanbiuyoe | 


ysuls ‘dW 


‘9BLPOS 9p BUIDYDS UN Pp UONDUOJ UD sagscyeuR sind onbysusew spueq Ans saensIsa1uS 
}2 plesey Ne SdISIOYDS SMIIAIOIUI PETZ ANS 19 OVL] UONeoydde | e sagisajur sduia) np soinseul sop 
INs JUSPUOJ as syeI[NS9I SIT “ZXZ [911019eJ uejd un suep suonsenb sap soipio xnop onb isure ues9g | 
ap suonjdsou0s xnep gsdjeur & uo a][anbe] suep urea) 9] Ins soUaLIadxe auN ,p syeI[Nsad sa] aJUasgId 
syon ‘gsodxa uos sued ‘SINOMAIAIOIUI Sop JUSWIDLOdUIOD 9] UAIq 19 [eq JUsOUENTJUT suOTsoNb 
SOP SIPJO | 19 URID9,| op UONdadU09 ek] onb JoNUOU nd & UO ‘aIIOJeIOQR] ap SadUstI9dxK9.p apIe | 
V (OVD IneyeuUIpso Jed d9}SIsse MIIAIOJUI OUN Pp IIPed J] SULP SINAMAIAIOIUI Sap jUsUTaIIOdUIOD 
3] Ins UdDJ9x9 suONsonb sap a1psO,| 19 ULID9,| ap UONdaduUOD e] aNb aouantjur,| ouMUeXa syn 
“INBL9 [| Ip PURLIVA PT 19 JURPUOdEI NP NO INIMITAIOIUL | Op SonbyystgjoesIVd sap e a[qenquije 
uOTVLUeA eT ‘UepUOodal Ne NO INIMITAIO\UI | B ado1d UONLILA k] 9JUD UOTOUNSIp ouN JUsssI[qQeI9 
SInoyne sa] aonb ysa ai3a}e1Ns e] op JUeLOdu yoodse UAV ‘sINILIgIUR XNeART So UOJ a] ou onb ayjared 
asuodgi-uou e] JUsdUENTJUI INb sinajoRJ sap yo[duio9 snjd ndsode un jUaJo sajepoul singq ‘ayjared 
esuodal-uou eb] ap apnyg,| @ anbndAyeue anbruyoa) ajueyoduit oun yUdISQIUI IpPfaAsoo’T ja AlayoIg 
‘ZUMINF-UdsueHY Op plepur}s INd}eUII}Sd | Op UOTOUOJ 
Ud UOISsois91 Jed 39 yUoTJONb Jed satjipour sinayeUINsd sop SATJLIAI 9LOVITJO,| JoIpMe.p youLed 
onbuiduis apna ayned sup ‘sinayeuINsa singj 39 SAOURLIRA So] JUSJUaSId STI ‘sn{d aq “QoURTILA PT 
JOSTUNIUIW & UOSeJ op ‘s]INOD ap suONoUO] sap side .p ‘axlf asvUUOT[NULYIE ,.P xNe} 9] 39 SUOT[MURYD 
Sop o][Ie} e[ Inod sayeurndo sinayea sap JUSUTULIAIEp S|] “‘UOIsseisaI Jed yo yUONONb sed sinoyeuNse 
SOP SIIJIPOU SUOISIAA Sap juasOdold sinaine So] ‘eUI9YIS 99 JURAINS ‘OXIJ BSeUUOT[NULYIE,.p xNv} 
UN,p UOTDUOJ US aseyd oUIaTxNap e] ap SINOD Ne NedANoU ap s9sO.Ua\UI JUOs INb sjuepuodg:-uOU ap 
UO][ULYd9-snos UN jULLIOdUIOD ‘alqnop aseUUO][NULYIE ,P BUUSYIS UN JUSIQPISUOD aa Je JOJEYO 
"S9IUI0D SOP NLAAIU Ne [ISSIG Nv S9SVUDU Sap NUSAAI oJ INS SoguUOPp op apre,| & Sonpgo0Id say 
JUANSNIIE S|] “SOUDIZJUT,P [NO DUIWOD JUSAIAS UD,S 19 SBJQPOU Sad Inod sqqIyH ap aseuuO][NULYIE | 
INS sjielap sap juajUasaid sinaine sa] ‘ailojeg[e ‘[eSuI OXIJ ‘[ed9 OXI :SPUTLULXS JUOS BOULLICA 
aP SI[QpoOul SIOI], ‘UOISIOgId LI Op oINSOU SUUIOD astid 3S9 LIOLIA\SOd B BOURIILA BI Jo “LOLIa\sod 
® QUUSAOUL LI Op SPIe,| & IDUINISS 1S9 JfeUOISa1 QUUSAOUL LT “SO[BUOIZ9I Sa[qeLIBAOD ap JUdA|e39 
yuapuodep Inb sasoywele UOISsaIZ9I op SanguIeIed sap JUSOWUPe Sa[QPOUL SJ] “SayBUOIsaI SUOTEUIT]SO 
so] Inod xneaatu sinaisnid ve sokeg op sonbrysselgiy sojgpoul sop juajuasgid ovY 39 NOX 


o1guunu 8d sueGg cel 


LE 


“syueLIOdUIT DUTLIOD SAIBPISUOD XN9P SNO} JUOS sagsTTeU 
-UOSTRSAP SUONRUINSS Sap Jo DOURPUD} OP SUONRUSA Sap [ONsUSW JUSWASULYD J] 19 NRIATU JI IS 
a] qeuuostel stwo1du109 UN jSd «STOW Q ap [e}0) uN nod ‘7 Inod suep ‘z Inod sioy ‘Zz Inod suep» [1yod 
a] onb yuowlaye39 Juanbipul sinajne say] ‘OseUUOT[NULYSI | Sp OULLILA P| DINPoJ NDA UO | oNbsJOT 
a[qQeiagjoid Jso «SIO g op [e}0} UN Inod ‘[ snod suep ‘Z 1nod sioy ‘{ Inod suep» JUDWIST[aANOUII 
ap [lod oJ ‘SOANNIVSUOD SdOURPUD} OP SUONLUITSS xNap IUD DOUdIOJJIP L] 39 [oIOURpUd) NeOATU 
3] UOS Sad sanbrystjeIs Soy Is ‘SIOJOINOL, “SogsTTRUUOSTeSap SUOTVUINSS Sop JONSUSI JUoWASULYD 9] 
ySo JaskTeue & g[9 onbuysnejs ev] Is Sa;qeuuOsies JUOS «fp Inod suep ‘p Inod sioy ‘p Inod suep» ‘«g Inod 
suep» ‘«g Inod suep» JUSUIAT[aANOUA ap s[tjoid sa] ‘UoUIAsUDDNY NP VIWTAV-LI-X 3° LI-X uouest] 
-BUUOSILSEP Op SapOYEUI Sa] SSI]WN UO .NbsIO] SasepuUOs sap Ins saapuoj sonbisopouoIYo satigs op 
SONUDIGO DOULPUD} OP SUOTIEIT]SA Sap Jd SIAST[PUUOSTLSaP SUOTJLUINSA Sap oseUUO][HURYDE ,p DoULILA 
RI ANS JUDUIDYONLADYO OP Sa[QPoUul SJUDIPJJIP Ip SUOISSNdIJEdaI Sa] JUOUTLULXS [9901S 19 UDI 
“QOURIJUOD SP SIT[PAIOIUL SINITIIOWI SI] 39 SITTIGeIS 
QIN [IOW VY ‘steIg ap SUIOUL J] JUSWOdUIOD INb oULLIvA P| OP SUONLUNT}SO So] JUDSSTUINOJ SaQUUOp Sop 
puadap a][te} e] JOP saT[eAJoqut say onb onbiput onburdwa apne ina] “SHIN QoueUPA ev] op soxouuo0d 
SUOTJLUIT]SS S23] 19 SOULIPSU SO] INs ‘saguUOpP sap puUddap aI][Ie} e] JUOP SoT[VAIOJUT Sap DOAe JUSUT 
-odnoi3 un .p 39 OXIJ S][Ie) Op SoT[BAIOIUT DdAv JUSWSdNOIS UN,p ‘(UOT[HULYIE | 9p oUeIPSUL PI SIIP-e 
-1S9 9) JUaWIAdNOIZ op soUaSge OUN,P S}aJJo Sa] JUaIedUIOD ULUBIS 39 UOSdWIOU, ‘aoe Ina] sued 
‘QUEIPEUL eT JURLOdUIOD aT[eAIOIUI | Inod are9uT] UONRTOdIAjUT DUN JaSI[NN & 39 SJBIOSIP SoT[eAIOVUT USD 
SONUTUOD SaguUOP So] Jodnols & d}SISUOD UOTLUINSS Pp spoyJoUT aNe UL: “[Nd[ed ap gyoedeo ouULIOUS 
QUN JUSANOS VBIX9 UOT[NULYIS Pp SOURIPSU Sap JOOIIP [Nd]ed 9] onbsind yueyodu Ja1NUO0S sueTqo1d 
UN 8] 1S9,D ‘SOURIPSUI Sap aseUUOT[NULYSE Pp SULLA P| Op UONeUIT}SO | Inod (s1JIpow UOT[HUeYDE 
-Wop) SHIN UoNNedel op spoyjow ve] e jadde juoj onbnsyeis el op xneoing xnoiquiou 39q 
‘SdD PJ & JUOWIDATILI[A1 SULIO} 1INOD & SJIPITLA BS INS SayURssolg}UI sonbuIduia saguUOp sap 
juajuasaid sineyne soy ‘AOYIeJA] Op asauodAy ,| op UONeULITJUOD PI ISO asATeUe 91199 op onbNLIO yoodse 
UA ‘SdD BI B Sasuodal sap opnjoexo | Jon[eAd Pp USAOW SUTLUOD 9]U9}R] DINIONNS vB] Op SUUDIAOWCUI 
OSATRUL | OP DUNN] SP INDJVOIPUL UN DUILOD DdI9PISUOD Js9 S}e][NSAI Sap I[eqO]s BoUdI9YOS eT] 
‘(SdD) Adaing uoneindog JuseND ke] ap sped a] suep satjoe UONR[Ndod ez] Ins soyfensusw saguUOp 
Sap juRssIUINO} sjoued sap Inod asuodal op Ind ,.P XNe} SI] JUSUT}SA S]I ‘APOYIOU 99399 Op ple | 
V ‘oarjoe uoneindod ez] Ins sa9uUOp Sop SURP UONPOIJISSe]O OP SINDLIA Sd] JOUIIO op UIJe VIATOR | Op 
SIA-B-SIA UONENIS ap SJUSWIASULYD SZJ INS AOWe] ap asempodAy | B sInOosaI UO AJBYsNg 19 JOWIg 
‘IQ! .P 9[qeHea 
RI 32 UOTJOEIVS ap sayIqeqold sa] ayUd UOLI[909 oUN 9)sIxo [I ,NbsJo] asepuos ap ued op 3904J2,] 
op uoneutxoidde oun ajuasgid 1adueds ‘uorssaiSai red uonejndod ej ap uonejuasaidal oun _p opre J 
V ‘1Q19IUL,P I[QULIA P| 39 UOTJOEIVS Op SouIqeqold sa] a1yUS UOTJe[I1I09 oUN 9)S1XO [I ,NbsIo] UONeI 
-9puod | & sajqenqune o8epuos sp urjd op sjayjo sop UoNRUINSa,| op aUIe[qQoId 9] spore Js0Uad¢S 
‘g]qeiousi sed 3sa,u asuodgi-uou e] puenb sinaia sap 9iyeuOU ap asauiodAy | 
OP SUPA PL] IOLIVA op jueyowUed s[dwiIs spoyjeu ouN jJUsUIATe39 ssodoid |] ‘soUR,;quIdSIvIA 
op WINUITXeU NP spoujeUl ev] op sed a] sed jsa.u INb 90 ‘JgIQ}UI.p sqeueA ey] JUeNbidu slapour 
Np SINdLI9 sap UOINQLIsSIp eT Jatyloads ap sed a81xa,u a][a nb aplos aI[9} op 9}INSUOd 3s sT[9 Nbsind 
giyeuLou op ssaujodAy | & Woddes sed ajsnqol dalapisuod jsd opouaW VY “aIpuodgl ap y]Iqeqoid 
eR] Inod a[gpour uN Jd 3919}UI,p a[qeLIeA ve] Inod a[apour un yarnbas agsodoid apoyjeu VT ‘apquiousi 
-uou osuodal op owsiursew un ajjedde uo.nb giajul,p afqeueA ve] op puodep inb osuodai op 
SUISTULIOU! UN & OP SIeIq J] DINpel op yuRWoWLIEd UOTFeUINSS .p spoyjsu suN dsodoid juoWINeag 
‘sapueis snjd aseuuo][Hueyse .p soqun 
SOP SIN[OUI Pp S9OULYD SOINI][IOW Ap UO SajQ[dwIOoUT sajsI] Say anb ye} Np a}duw0d judUUIT saTapoul 
sine ‘sn{d oq “asepuos ap saseq sap yoduroour alg}oeIVO np a}dui09 aipuas yueAnod satidoidde 
uone[ndod ap soijjiys sop juasodoid sinayne sa] ‘altejogre aseq oun jd Saye[duooUr sayst] xnop 
QUIQUIOD UO | onbsIOy [e10} UN,.p SUOTEUITSO Soy DUTILX B[NUL Jd YOOT[O ‘sourey op spore /T 
‘Ysry Inassojoid np sanbysyejs suoesi[eal So] 39 SIA 
Rl INS 13d[[94 UAT,P UOTX9]JoI oUN OJOUINU JUaSaId Np INgap Ne vIdANON UC ‘ajanbua, p sanbiuysa], 
op slesIoAtuUe ,¢Z 2] JUeNbIeU OIOWINU ZI SUep Ue UN ke K {I aI[qnd sonbiydeisowep sajgnbus 
SOP UOSIVUIQUIOD P| 19 [NUNS JJ ANS JfDITIe UOS BUZIOUWI} Ud SUIWIOD ‘sanbiso;opoyyeu yo sanbysyeIs 
SO9PI SO][OANOU op JojUasoid Op snUNUOD & Ysry Inassojoid 9] SIA es op UJ eT B.Nbsnf ‘ajqenbrewl 
ISOYD “SUP 06 OP 93k | B QUWOJNe 139 Spgoap Iso INb ‘Ysry sI[se] & sIpap ISO OJQUINU JUasid JT 


O1ITUNU 9D SUBG 


epeuey enbysieys 
ZEL-LEL “dd ‘| .U ‘9% JOA 
000zZ aiqwes~ap ‘ajyenbus,p senbiuyoo | 


- eh MSHS PHAR Du: NT, ANTI IA StNy CUa IEL ye Efeeh Upset 


ke Ret i - 


‘guaran 


He he = 


“yyy 


iy, papa pata 
Kt OGRE SES Bors 


PSB E DED WV IUSS 


pO Be ange. p Ue TOE 
(er An 2 qhes bai: Bitte aybas cates: 


LECHUHINES BKMOR ET 


ve 


mits ae HES iar C2 S(T erry Unis 


hiettaea-64: fat 
Sass teteroarn > tae Domagattine (HGRINe ww ne ga cunteytgc 


he in denon kaye pls “tr ioe (Reais . a ee ee 


i 
‘ aay ake 


apaaraitons ape yt CONTE OF Eyes vee Lie ia icee home: qae 4p ilee igs ge fie fmnalygres naen 


URE A Nee a 


1 


fae 


ee) 


(6 5 ARR as a Re ae ea caer De ee rie cape as ST SJUUISTOIBUIOY 


7 ures} 9] Ins a9uUOUE UONesTAN,p 
QUIOvy op souatIgdxa uN p s}eI[Nsay ‘OVI 2[Npoul un suep suonsonb sap sIpso | 39 SUvIOD Sap UOTdINU09 BT 
SHONd “W 
GCIZ) 2 ea ee uoneorydde sug :joued sed sajygnbus sap sed 3] SUBP INIMOIIAIOJUL .P S}OJJO SOP UONSI[SPO|] 


LATHASOOT'D 39 AMAA Id ‘f 


11 Mr nd ee me ei ECO ak aah cs aoe CaCO Gt AOC G8 oo Nad Scie si sjuepuodgs-uou sop 
aseUuUOT[NUPYE-sNos ddAv UOISsoIB9I Jed no yuaHonb Jed uoneunsa | Inod saseyd xnoap e sdeuuo][Nueyoy 


dq 1TH? YOAVAO OA 


L6I afro, fe) ay), ei7ai fo! a) wire fetal fal (em) ae) 6. 1e1/4, © ef a. 6, a tattel ve) eer ellos Te: ib wiieie! fo Ad felis 'aaal jel fede Kose! (0) 0: elceeiia) tonne, XNedAIU sInaisnid ze So]Qpoul op apie | e 
SUOIs91 Sayned Inod sauuahour sap onbryssesgiy suustsgkeq UONeUINSA 


OVa ANTI NOA A 
ee Jouepud} e] ap suoNeUIPISe sap 19 


sagstfeuuostesep SUOTIEWINSS Sop VdeUUOT[NULYDE ,.P JOURLILA LJ INS JUDWII|[2ANOUAI ap suL{d SIOAIP Op 19JJO,'T 
THHLS “9'd 1° NAAV PW HO 


o5 Gee SONPUDA SUOSTLUL Sap SULIPEUI 9jU9A ap xd sap sonbrydgi Jed soueLeA e] op UONLUIISA | }9 UORUINSA "J 
NVWOIS ‘S'a 3? NOSdWOHL fH 


LSI Boe ie) “ayNe) 6) a fei. (6: 6) bo sec Le) 0) @yioy 1910; /o@iLeneine Ke. 6) Sh eMe. 6) any ofhh eh ear ie\.9) ol 4 aie 6: a), 6)selieueNe_el ene bie) isiiehen is] istic daT}0R uoneindod | Ins soguuop sop 
UONVOIJISSEID Op INIA | <p UONeUINSA | INod ajUa}e] INjONS op DUUDTAOYIeUl asATeUL | OP SUPE A 
AYAHSNA Wl? YHWAI ‘dd 


€SI 6) 0/00) 0 Tol eda tants, (ofa) /sieike: eel (ae) (o7 @velsn ible etm. in) inl als jetta” aike! elie) dueb)ele.s('e isin) ele) elel= ‘= uon}oaIasS op soiiqeqoid S93] 1) SOINSOUI So] onuo 
a[qIssod UONRIILION ap seo Ud a[eZ9uI UONeJgpuOd oun Inod Jnewrxoidde osepuos op urjd op jojjo uA, 

YHONAdS ‘d'a 
CP ee ee ey ite eae aia ee 2]qQeJOUsI-uOU dsUod9I-UOU op adUaS9gJd Ud UONEUINSS Pp SpOYIsU dU) 


LNOWNV4d ‘of 


SEI O) 4) 46), 6 (ele) 1a! 4, oe oe eicelte) alib) ouSt 0 \etrel’s io malta ete ale le le aWalletielen¢ Nel ied etatel sie: 'e ie] oe ie "eleneie; (te) eie1/e'.0 sougs019}94 UOISNIOUI .p souiqeqoid 
DOAB $9}Q[dWIOSUT $9}sI] Op seit SUOT[MULYDE Sap Inod uoNe[Ndod ap saijjJIYO sap 39 aIIIe} VP] Op UONeUINS| 
VIALNVd ‘D'S ?? MOOT1Od ‘HS ‘SANIVH “A'd 


3 eee JUOUIANOAYP op 9IA dUQ — USTY SSI] 

IDATTHS ‘dT 

1 oiguinu 29 sueg 
SAYaILVW SAd ATAVL 


000Z 21qQuIZ909p ‘7 O19UINU ‘gz SUINTOA 
epeues onbnysneis Jed s9}Ipe ond ou~) 


ALANONA.d SANOINHOAL 


“oan pee 
aka a 


re 


23% mee a 40.1 Shed Rea mapieameUmar’ Stn 

wie: nae ene eae 2 Ae wo 1? 
Cy ne, cg 

ies eens wh en eed nutans: taf tin Spgs era} ee 

Np MRS G/N HG ST dail ideas Meas Sis ‘vo ae 

ne PE I Fh > ae Zi Oa; Ae ib ee Mee SEK: ies . 

Geepes ve erat etn an He ee wip: aes chien get So, 7 <3 


{ e cy x tf 4 A ee ay ou 
pe 4S — —* a Wf, >? ive as a. 
sing Sadngid we ae = se ing 5 mao atc ar a Psy Wiser (oe ale) wives 
Fis MEYER ORT - ritayt Cet Se wah fe ht <oe wm) asters 
; ; OSS «eo my ‘ rate) aties) we? < 
_ SS CET Ae } £ ae vk 
ne¢ wr) a xn io U 
6 tay, i¥ as | : 
U A y f +3 Fa re 
o 2 pure ° 
¢ { 1? 
- or Fee? are! es eens ee ee 
<— . -¥ 
“4 yan re SIME HOM FOU IOND a : b nied 
ar eee Sar tet Gmc mtr Sr ey OSWeT- Une Pon c 
; b »% “ » Ae " r ry A s 
Be GONE INTL nie Sona Ge abi setters nits i. 
a wel an 4 oa 3 ‘ . 
ae WS) Siew Hecate he sat a i a : 
i : t ; 
T AoE ee fh med 1 eat Ww 4 it ‘ 
a : 
i] tt 
a oh “rue ; 
ne T¥ i i 7 
A ee ‘ <J WS : ror ty t r 
: ’ » 
Tage | mai ' ¥ ‘2 ‘ R ‘ 
* v'? 
Boe bole fest i ceyey 
ye et Tene : ere ae > of eee oS ae Ter eee ‘ i 
7 my r — 
aah (haehe somes tor 


a> 


ea gy OAT ri © eicee “ch 


7 : 
if e* i~* 
Fr 
; aA) ges | id 
ar’ wi] 
s 4% ay 


. , 2 ee, coin 
Ni Lt “7 A 
of ot eie tees gpl IM Uigereleeney bike was 


aa! a eee Le RN Oey, 10 ey 
pints ame. ve 6 veek sani me Ory wave uo Senge, 


7 Wits = 4, Se be, ; C28) 41) Sa78e wy ald it) - 
' i} . A ware i 


BiSIONA SF TUNE yeET E> a a 


‘dnoonvag inod uonesmdsut,p 
d0INOS OUN ‘JUdINeUWAp Ja ‘339 UO sanbnstye}s xne 3a aygnbua .p 
SIZO[OPOYISU P| B Sa]UeLIOdUIT SUOIINGLUOD Sas 39 SIUNUWP Sa] 19 
sguuiddo sa] nod apuojoid uonednosogid es ‘asnaise}U0d SIAIA 
ap a10f Bg “YShy SISA] ap aTOW9UI FT & 9IPEp ISA OIQUINU a_D 


(0007 - OT6T) 
HSI AITSaT 


"99qQ9NQ NP susIONe}s Jo SAUUDIONSNeIs sap UOIDOSSY | 
Jo epeueD np onbnsneig gigioog ev] ‘Yyoreasay uorutdG o1qQng Joj uoNeIDOssy uROAWY,| ‘slgnbuq,p suaronsneis op 
s[eUONeUII}UT UONeIOOssy | “‘UONeIOOssYy [BONSNe}g ULOLIOUTY ,| op S2IQUIOUI xNe Lajjo }s9 JINpe1 xud UN “eo'URDIEIS@ JapIO 
‘PHINOD Jed nO H€/6-688 008 I Ne Inardoogiai sed ‘EEQI-OOL 008 I Ne suoydsta) sed zapueumOD no 910 WIM 
epeued ‘(oleUQ) EMeNGO ‘oepyseg onuaae ‘QZ] “UOTE[NOIIO ev] Bp UONSay ‘uOIsNjsIp Le] Op UOISTAIG ‘epeueD onbnsneis e 
JUSUSUUOGe Pp spueUEp aNOA MuaAred ose} op QU *(seme|duiaxe Z x $ OL) VO $ OZ ‘shed sanne ‘(samejdusexe Z x ¢ 9) WD 
$ ZI SUp-sieIg :epeueDd Np Inatig}xa| & sloAua xne Juonbidde ,s sasrequouta[ddns uosresay] op ste] soq “souustpeued aU9A 
9p Sexe} sa] sed snjour.u xd a] -aguue Jed WD $ Lp ap 1se (enZoyereo ne GqX-100-ZI .U) sIanbue,p senbruysey, ap xud oJ 


yusMIINNOGYV 


“Inayne | B SpUINO}aI sed JUOJas aU SosTe[dUIOXe S3_D “onAdI | SULP SagjUasgid SAATIOIIP So] UOTas ssrydesBO]AOep 
saire[duroxa anenb JoA0aua Pp 212g ‘9LO VIX epeued “(oleuC) emeng ‘aimseg s Aouuny ‘epeues anbnsneys ‘se8eusul 
sap sgidne sajgnbua .p sapoy}sul sap UOISIAIC ‘YZUIS “q'W ‘W ‘Joyo ua Inajoepal ne steduesj Ud No sIe[sue us 93Ipe1 21x2} 
9] JtusAred osre] & S9IAUI JUOS J[OTUe UN aNIesed aM] jULIISSp sINo\Ne sa] “uUL_] SIO} xnep set{qnd jsa ajgnbus_p sanbruysey, 


dNAdJ e| nod sajxa} op UOIVe}UASIIg 


‘epeues onbrsneis op lu uoTdepel ap s1TWIOO 
NP So[[2d JUSWoITessad9u sed JUOS dU ONAdI BR] SURP SasTUID suOTUIdO sa] 19 3}x9} INZ] ap NUD}UOD Np sa[qesuodsal JusINeWIAap 
SINIINe S2| STRUT ‘oNbNHLD ouUN zk STUINOS JUOIOS SITE SI] SNOL, “sa][991 saguuOp sap e seenbijdde no saguuop ap 219a]]09 eT 
Inod sagst[Nn 939 JUO Inb sapoylaul ap UONeN[eAY | & 19 UONLIOGE]E | & 2epIODOe Isa oJeT[NoMed souryodu sup “sastpe1ouss 
SoulgisAs op yuoUIaddojaagp 3] 19 SagUUOP ap asA[eUe .P 19 UOTEUIN}Sd,.p Sapoy}su sa] ‘sonbnste}s saguUOp ap UONeISIIUT, | 
‘sonbrydessoulep sapnia so] ‘uonesifeuuostesep P| ‘sonbrzojouosyp savas sap asdyeur | ‘ajanbua ,_p sapoylsul sa] Ins ayssaysel 
RI ‘sajgnbus sop uonenyeag | ‘saygnbus soy suep sInad so] ‘9}9a[[09 9p SapoyJsU ap 19 SaQUUOP eP sadINOs salUaIeIjIp 
op uonesr{nn | ‘onbryeid a1pso .p sajyurenuod op jue[Nodgp uoNdaou0d ap soulgjqoid say ‘ajdurexa sed ‘ouruos anbysneis 
dUISTULZIO UN jUaSsaIg}UI INb sanbNsnejs sapoyisUl sap s}oodse SIOAIP Sa] Ins sajonse sap aijqnd ajgnbua.p senbruyoay, 


NOILLOVddae Ad ANOILITOd 


ppounZ anbysiyojg ‘jayMsg ‘q 19 BunX ‘AA ‘[auey “H “AIq “q ‘uounesg *4-"f sjulofpe sinajaepay 


soysijnjg fo nvaang upijoajsnp ‘aiseury “S 


Ayisdaaiul) pavadDy] ‘KYSAR[SCZ, “VW 

dua Yyoapasay uoiuldE [DUOIIDN ‘INfOM “W'S 
‘QUT ‘Say ‘BIOQSYe AM ‘f 

‘Uy WwIsay “WeryyeA “Y 

Ayisdaaiuyy ajnjg o1yec ‘Kuseis *y 

uojdupyjnog fo Aj1saaaiuyZ) “IQUUTAS “f[°D 
AJISAIAIUL) 4BSDAY UOUNY ‘I9NIS “Y 

IINJISU] UDGAL) AY] “U2ZINIYIS “f° 

[DADT ajtSdaaiuy) ‘\S2ATY “d-"T 

Ayisdaaiuy) uojajavyD ‘Oey A N'f 

Ajisdaaluy) MadgaH] ‘UURULIDSJIg ‘C 

ppounZ anbysiojg ‘SuIoN *d 

ands] ‘so1stnjg fo nbaang joajuag “ueUeN ‘DO 


ysuls ‘dW 

hoy “d 

ynoouey “7 

(Quaptsgid uarouy) 4aIeI1d “A 


ujooulT-vyspigan fo (issaaiuy) “UrYeT ‘d 
INALIG SINSYDIS JOANIJNI1ABY JDUONON ‘YOY “d 
‘QUI ‘JoIsSay ‘UOIeY “H 

YN ‘ao joousiplg jo4ua ‘OH “dd 
ppoung anbiysij0j¢ ‘no[solply “YW 

ppoun, anbysijojg ‘OuIquIeyd *£ 

AjISABAIUL) 3IDIS DMO] ‘I2][0 “VM 

Ayssaaiuy) WP soxay ‘eBunq “f 

FASNI ‘AAP ‘D-'f 

snsuag ayi fo noaing ‘§‘Q “ARID 

ppoung anbysyojg ‘Iopulg “Wd 

JINj1JSU] ajsuv14] YIADaSay ‘IBUIIg ‘d 

O1dDJUC) Udajsay fo Ajisdaaiuy) ‘asnoyl[eg “Ad 


sg1d0sse sinajIepey 


ppounZ anbysyojg ‘ysuts “d'W Jeyd Ua IndjIEpIy 


NOLLOVdadd Ad ALINOOD 
yume OD 
(uononpolg e] ap Ina}de1I1q) epkey “4 
sIPH OLD 

Jopurg ‘Vd saiquisyjAl 

guol\syorlg f° yuapIssig 

NOLLOGYIGC Ad ALINOD 


“spoyay SANeITEND UT sJUsJUOD [euINos 39 ‘sONSHeIg 


0} XOPU] JUSLIND suep saoues9J9I So] JOANON ua yned uQ “AjIsIOATUL) SnUIseIg ‘ASO;opoyTey] YoIeasay [BIOS JO esequied 
WS 19 sloensqy spoma pue AJoayy, eonsneig ‘uelonsneig Aoamng oy], suep agto}adai 3se a1gnbus .p sanbruyse, 


epeurd onbysye1¢ Jed a9}Ipe enAel sup) 


ALANONA.d SANOINHOAL 


epeury) ice fie? WN 


fo | 


emeno 
Sv00-71Z0 NSSI 
[|OWIJSOWAS ‘9DIPOLEd 
enbojeyeo ne GdX-100-Z! oN 
L002 Je1A94 


"910 VLM Bpeued ‘oUe}UO ‘eme}C ‘epeueDd enbiyjsie}s 
‘HunayseW NP UOISIAIG ‘80U8DdI| ep S}O1p sep 
UOISS8NUOD EP SEdIAIAS Sap ajqeieeid 909 UOl}eESUO}Ne,| Sues 
UaWeIANODE! Bp sua}sAs UN SUeP JeUIseBeWWA,| ap No ‘aJjne no 
‘enbiudeiBojoyd ‘anbiueoaw ‘anbiuosjoe|9 UuoNOnposdes ‘enbyeubew 

yoddns ins juswaujsibaiue ‘ios 80 anb uaAow anbjenb sed 

NO 9@wWJO} enbjanb snos ‘uoleol|qnd ajuaseid e| ap nuajuod 9| 
OJ}JWSULI] BP NO a1INPOIdas Bp YPsajU! }SO |] “S@AIJBSOJ S}OIP SNOL 


L002 ‘eLISNpU],| SP SSI 5 


epeueg enbijsiejSs ap ajqesuodse 
aljsiulw 9} Jed eesuoine uoeognd 


Cc OYWFWAN © OQCUWNTIOA © O00? FAYAWASDAG 


VAVNVO ANDILSILVLS UVd 
TALIGa 
anagy JNn 


ALINONG.G 
SAINDINHOAL 


epeuey) Steg obteg HN 


Eel 


eMmelO 
Sv00-71Z0 NSSI 
BIJOUJSOWOS ‘9}HOIPOWOd 
anbojeyeo ne GdX-100-Z oN 
L002 498uA94 
"910 VL Bpeued ‘oUe}UO ‘emeyO ‘epeued enbijsiels 
‘HuljeyseW NP UOISIAIG ‘8DUBO!] Bp S}ioup Sep 
UOISS8NUOD EP SEdIAIaS Sep ajqejeaid a}09 UOI}eESIO}Ne,| Sues 
‘Ju@WeJANODe! ep ewa}sAs UN SUepP JoUISeHeWWA,| ap No ‘aljne no 
‘enbiydesBbojoyud ‘enbiueoew ‘anbiuosjo0e|9 UuoNONposdes ‘enbyyaubew 

woddns uns juewejsibaiue ‘os 80 anb uaAoW anbjanb sed 

no ewido} enbjanb snos ‘uoleaiqnd ajueseid e] ap nuajuod 9} 
SIJJOWSULI] BP NO aJINpOIdal Bp HPsaju! }S@ |] “S@AIBSOJ S}Oup SNOL 


L007 “SLISNPU],| EP SISIUI 6 


epeuey anbijsije}S ap ajqesuodseal 
aJjSiuIwW 9] Jed sasuojne uoHeoqnd 


Cc OYWFUWAN @© OCUWNTIOA © OOO0T FAAWASDAG 


VAVNVO GNDILSILVLS UVd 
AALIGa 
anAgy ANN 


ALINONG,G 
SINDINE aan 


epeueo epeuen 


BpeuUe ) sosieys anbysieis 


AT LINONGG 
SUINDINHDAL 


ee 
ae 


aR 
q me ac 
a ” 


rid 


We 
ne 
ni oe 


uh any raya] Cul . 
ML We Meas a me Pea ik a) Ain a ae Ae (if 

iho " ‘ae jul +) is ah y eh LAD Serie » "i C ' v . 
, : ee vii Tt) oe on eee \ FAS eT ee te : tre aby avy) 
PA i Sa a hele, hu a fl WN A, Bd 1 bs i | Rs ey 
Ue ay, oe ae i By M | are a ae } ; ) 


ay he uae 


f i ie ip i> 1 Bi eae] Pye? Pt = ' 

Wat Be May, ue ah NAMA leloa's uk Wea 

“Ail ; Tey ie |, Nt a eh ie ‘ an ey rT mt * ie ine Ph ACs, ; wy) hay AY ; 
A ipa) i aly aed ie hi iM a A if se i i Wh | 1 \ y : } ¢ 7 


ct 


ty 
ie 


fe 


SSS 


at 


if 
i CN 


H| ia 4. 
Hh 


eee fas 
Ue 
ag 


ant 
Cer neta 
ns 


Way tee 
‘ oa), 


